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ABSTRACT 


There  has  been  a  strong  tendency  in  the  foreign  language 
education  community  to  encourage  learning  vocabulary  from  context  and 
to  discourage  direct  instruction  of  vocabulary,  this  despite  the  fact 
that  the  amount  of  research  on  vocabulary  acquisition,  particularly  of 
older  learners  in  a  foreign  language,  is  relatively  small.  Recent 
research  has  illuminated  potential  advantages  to  more  deliberate 
instruction  of  vocabulary,  as  well  as  increased  use  of  dictionaries. 
This  study  examines  the  combined  effect  of  direct  instruction  and 
learning  from  context  on  an  authentic  German  text,  looking  specifically 
at  subsequent  level  of  word  knowledge,  response  times,  and  recall 
protocol  scores. 

The  study  used  cadets  studying  beginning  and  intermediate  level 
German  at  the  Air  Force  Academy,  and  processed  them  through  a  four  day 
cycle  which  included  familiarization,  vocabulary  instruction,  reading 
comprehension  tasks,  and  then  vocabulary  testing.  A  delayed  test  was 
given  two  weeks  later.  The  two  independent  variables  were  whether  they 
had  received  instruction  on  the  targeted  words  and  whether  or  not  they 
had  access  to  an  on-line  dictionary  during  the  reading  of  the  authentic 
texts.  Verbal  skills,  as  determined  by  ACT  and  SAT  scores,  were  used 
as  a  covariant.  The  data  from  one  hundred  randomly  selected  cadets  was 
submitted  for  a  MANCOVA  analysis. 
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The  results  of  MANCOVA  provided  some  findings  that  serve  to 
reinforce  much  of  the  more  recent  research.  Direct  vocabulary 
instruction,  even  in  its  most  basic  form,  can  further  vocabulary 
level  of  knowledge  above  that  available  solely  from  contextual 
learning,  or  contextual  learning  with  lexical  access.  In  the 
absence  of  direct  instruction,  lexical  access  is  capable  of 
significantly  improving  word  knowledge.  Lexical  access  also 
provides  an  advantage  on  reading  comprehension  tasks,  though  this 
factor  did  not  reach  statistical  significance.  The  examination  of 
response  times  proved  inconsistent  and,  ultimately,  inconclusive. 
Verbal  skill  did  not  prove  to  be  of  consequence,  but  it  must  be  kept 
in  mind  that  the  student  population  possesses  a  high  and  relatively 
uniform  level  of  verbal  skill. 
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CHAPTER  1 


THE  PROBLEM 

Introduction 

The  study  of  vocabulary  acquisition,  traditionally  regarded 
with  less  enthusiasm  by  researchers  than  the  study  of  grammar,  has 
enjoyed  a  recent  resurgence  of  interest  {Beck  &  McKeown,  1991)  . 

This  increase  in  research  stems  in  part  from  the  central  importance 
of  vocabulary  in  second  language  learning  and  acquisition  (SLA) 

(Crow,  1986;  Nunan,  1988;  Laufer,  1986;  Henning,  1975;  Koda,  1989), 
as  well  as  the  development  of  new  cognitive  models  that  provide 
potential  explanations  for  the  development  of  word  knowledge 
(Bialystok,  1988;  McLaughlin,  1987;  O'Malley,  Chamot,  &  Walker, 

1987;  Hall,  1992) .  Despite  the  growing  body  of  research,  many 
facets  of  vocabulary  acquisition  remain  unexplored,  and  those  areas 
that  have  been  investigated  focus  largely  on  children  learning  their 
native  language  (LI)  rather  than  adult  learners  of  a  foreign 
language  (FL)  or  second  language  (L2) . 

The  existing  research  in  LI  makes  it  clear  that  the  vast 
majority  of  words  acquired  by  native  speakers  are  not  learned 
through  direct  instruction,  but  rather  through  incidental  exposure 
to  words  in  contexts  such  as  reading  and  discussion  (Jenkins  & 
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Dixon,  1983;  Nagy  &  Anderson,  1984).  The  studies  have  done  so  by 
estimating  the  average  number  of  new  words  that  students  learn  over 
the  course  of  their  school  years  and  then  demonstrating  that  direct 
instruction  is  incapable  of  accounting  for  these  increases  in  the 
number  of  known  words.  This  has,  in  turn,  led  to  a  scrutiny  of  how 
students  learn  new  words  from  text.  The  results  indicate  that 
students  can  acquire  new  words  that  they  encounter  in  text,  but  that 
the  rate  is  fairly  low.  Even  this  low  rate,  however,  is  sufficient 
over  time  to  result  in  considerable  increases  in  vocabulary  (Nagy  & 
Herman,  1987)  . 

Reflecting  this  top-down  communicative  emphasis,  Coady  (1993) 
suggests  that  current  approaches  "assume  that  vocabulary  will  be 
learned  naturally,  with  little  or  no  overt  instruction,"  (p.  218) 
but  that  this  assumption  is  "problematic."  Some  problems  were 
evidenced  in  learning  through  incidental  exposure,  most  notably  the 
inability  to  guess  the  correct  meaning  of  many  words  and  the 
possibility  of  applying  incorrect  meanings  (Kelly,  1990;  Schatz  & 
Baldwin,  1986;  Williamson,  1989;  Haynes  1993) .  In  general,  attempts 
at  direct  instruction  are  successful  to  some  extent  (Mezynski, 

1983),  with  some  methods,  most  notably  the  keyword  technique, 
resulting  in  consistent  and  positive  gains  (Nation,  1990;  Pressley 
et  al,  1987;  Stoller  &  Grabe,  1993)  .  The  tendency  throughout  the 
studies  of  vocabulary  acquisition  was  to  examine  direct- ins truction 
and  incidental  learning  independently,  or  to  pit  the  forms  against 
each  other. 

Another  finding  in  the  extant  research  is  a  persistent  high 
correlation  between  vocabulary  knowledge  and  reading  comprehension 
(Curtis,  1987).  In  short,  a  knowledge  of  vocabulary  (and  the 
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associated  general  knowledge  of  concepts)  is  a  vital  factor  in 
comprehending  texts  (Anderson  &  Freebody,  1983),  and  the  ability  to 
comprehend  appropriate  portions  of  the  text  is  necessary  if  one  is 
to  be  able  to  accurately  acquire  new  words  through  incidental 
exposure  (Drum  &  Konopak,  1987)  .  Although  research  has  indicated 
that  vocabulary  instruction  can  increase  comprehension,  the 
improvements,  despite  achieving  significance,  generally  remain  small 
(McKeown,  Beck,  Omanson,  &  Perfetti,  1983;  Stahl  &  Fairbanks,  1986). 
Nevertheless,  investigating  the  effect  of  the  availability  of 
definitions  on  comprehension  may  shed  further  light  on  the 
vocabulary-comprehension  relationship . 

The  impetus  behind  this  study  is  the  possibility  that  the 
direct  instruction  of  definitions  may  bolster  subsequent  learning 
through  exposure  and  possibly  increase  overall  reading  comprehension 
of  the  targeted  texts.  As  stated  earlier,  studies  of  vocabulary 
acquisition  have  tended  to  examine  either  learning  from  context  or 
direct  instruction,  with  no  major  studies  considering  the  effect  of 
a  combination  of  the  two.  The  absence  of  studies  is  somewhat 
surprising  when  one  considers  that  a  combination  of  methods  is 
generally  recommended  (Graves,  1986)  and  that  this  pattern  of 
definitional  instruction  followed  by  exposure  to  the  new  words  in 
text  is  well  established  in  most  language  classes  (Blachowicz, 

1937).  Recent  studies,  most  notably  that  of  Knight  (1992),  have 
shown  that  giving  adult  L2  learners  access  to  a  bilingual  dictionary 
while  reading  a  text  led  to  significant  increases  in  both  the  amount 
of  incidental  vocabulary  acquired  and  retained,  as  well  as  greater 
reading  comprehension.  The  current  study,  which  also  includes  a 
variable  for  on-line  dictionary  access,  seeks  to  investigate  whether 
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there  are  advantages  in  more  deliberate  advance  instruction  of 
definitions . 


Statement  of  Problem 

There  has  been  insufficient  research  into  the  ways  in  which 
direct  instruction  and  incidental  learning  interact  in  forming 
vocabulary  knowledge.  This  study's  primary  aim  is  to  answer  the 
following  question:  Does  prior  definitional  instruction 
significantly  enhance  subsequent  level  of  knowledge  of  targeted 
words  from  an  authentic  text  above  that  level  achieved  by 
acquisition  from  context?  This  question  is  to  be  examined  under 
conditions  of  lexical  access  and  no  lexical  access.  Recent 
dictionary  studies  have  shown  improvement  for  subjects  having 
lexical  access  during  reading  and,  as  a  result,  it  is  important  to 
determine  whether  any  additional  advantage  can  be  gained  by  prior 
instruction.  If  the  students  achieve  the  same  degree  of  success 
with  dictionary  access  alone  as  they  do  with  the  addition  of  prior 
definitional  instruction,  then  the  common  practice  of  such  pre¬ 
teaching  is  brought  into  question.  If,  on  the  other  hand,  the 
results  indicate  a  significant  difference,  questions  arise  as  to  why 
this  improvement  exists  and  how  it  might  be  optimized.  In  either 
case,  new  information  will  arise  concerning  the  relationship  between 
prior  instruction  and  learning  from  context. 

A  secondary  connected  question  is:  Does  prior  definitional 
instruction  significantly  increase  reading  comprehension  of 
authentic  texts  containing  those  targeted  words?  Given  the  close 
nature  of  the  relationship  between  vocabulary  knowledge  and  reading 
comprehension,  it  is  enlightening  to  examine  the  impact,  if  any. 
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that  definitional  instruction  may  have  on  the  comprehension  of  the 
authentic  text.  Although  some  studies  have  indicated  significant 
comprehension  increases  because  of  prior  vocabulary  instruction, 
many  have  registered  less  significant  differences.  Improvement  in 
reading  comprehension  is  an  indirect  extension  of  the  new  lexical 
knowledge  and  can  be  mitigated  by  numerous  other  factors  (Bernhardt, 

1992)  . 

Distinctions  between  LI  and  L2  Learners 

As  mentioned  previously,  the  majority  of  the  data  concerning 
incidental  vocabulary  learning  is  found  in  LI  research.  Although 
providing  insight  and  general  direction,  the  results  of  LI  research 
with  children  must  be  applied  cautiously  to  adult  L2  learners. 

There  are  important  differences  between  these  two  groups,  some 
subtle,  others  more  direct.  Adult  learners  approach  L2  with 
concepts  and  vocabulary  already  established,  whereas  children  are 
lacking  many  of  these  adult  concepts  (Stoller  &  Grabs,  1993) .  Also, 
the  rich  contextual  sources  enabling  the  naturalistic  acquisition  of 
LI  are  generally  unavailable  in  a  foreign  language  learning 
situation  (i.e.,  television  and  magazines  in  L2 ) .  Foreign  language 
learners  normally  do  not  have  an  established  listening  vocabulary 
(Hague,  1987).  In  addition,  adults  normally  use  different 
strategies  in  learning  words  than  children  do  (Stoller  &  Grabe, 

1993)  .  Lastly,  mature  learners  may  prefer  more  direct  (traditional) 
instruction  (Chaudron,  1988),  which  often  includes  a  propensity  for 
consulting  definitions  (Stoller  &  Grabe,  1993) .  In  light  of  these 
differences,  it  seems  likely  that  incidental  learning  is  different 
for  FL  learners  and  that  some  direct  instruction,  already  an 
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integral  part  of  many  foreign  language  classrooms  around  the  globe, 
provides  an  important  link  to  vocabulary  learning. 

Significance  of  the  Problem 

Recent  Ll  and  L2  research  has  placed  increasing  emphasis  on  the 
possible  roles  of  vocabulary  instruction;  yet,  research  on  the 
effect  of  teaching  targeted  vocabulary  words  to  adult  FL  learners 
remains  largely  unaddressed.  This  study  investigates  this,  area  in 
order  to  provide  some  initial  answers  on  the  effects  of  this 
practice,  and  serve  as  guidance  for  future  research  on  vocabulary 
instruction.  Accepting  the  view  of  vocabulary  learning  as  stages  or 
multi-level  (Beck  &  McKeown,  1991),  then  the  research  provides  some 
insight  into  the  initial  stages  of  acquisition.  The  study  also 
investigates  the  relative  effects  of  prior  vocabulary  instruction  on 
subsequent  depth  of  lexical  knowledge  and  automaticity . 

On  a  more  practical  note,  the  study  involves  the  use  of  a 
computerized  instructional  program,  which,  in  turn,  provides  more 
information  on  the  use  of  technology  in  L2/FL  instruction.  If  this 
instruction  results  in  significant  gains  in  vocabulary  knowledge  or 
reading  comprehension,  the  study  will  provide  a  potential  format 
that  other  instructors  or  researchers  may  adapt.  The  SLA  field  is 
attempting  to  come  to  grips  with  the  issue  of  how  new  technologies 
are  best  employed,  and  this  study  supplies  some  suggestions.  In  the 
long  term,  this  type  of  investigation  holds  the  potential  to  allow 
students  to  learn  vocabulary  (at  least  the  initial  exposure)  at 
their  own  pace,  thus  freeing  class  time  for  other  vocabulary 
enriching  pursuits. 
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One  side  benefit,  albeit  not  an  actual  objective  of  the  study, 
was  to  re-examine  Knight's  (1992)  results  indicating  that  lexical 
access  while  reading  texts  provided  significant  differences  in  word 
knowledge  and  gains  in  reading  comprehension.  Another  noteworthy 
aspect  of  the  study  was  keeping  the  amount  of  time  spent  instructing 
each  word  realistic  (less  than  three  minutes) .  Much  of  the  previous 
research  has  not  concerned  itself  with  this  limitation. 

In  short,  the  study  has  significance  from  both  the  theoretical 
and  pedagogical  standpoints.  It  begins  to  answer  some  of  the 
questions  concerning  the  nature  of  relationship  between  vocabulary 
instruction  and  subsequent  vocabulary  knowledge  and  reading 
comprehension,  it  provides  further  insight  into  the  use  of 
technology  in  SLA,  and  it  provides  a  useful  framework  for  future 
research. 


Purpose  of  the  Study 

The  purpose  of  this  study  is  to  examine  the  effect  of 
instructing  college-level  beginning  and  intermediate  learners  of 
German  on  the  definitions  of  targeted  vocabulary  they  will 
subsequently  be  exposed  to  in  authentic  reading  texts.  The  measures 
of  interest  are  vocabulary  knowledge  and  reading  comprehension.  The 
research  questions  for  this  study  are: 

1.  Is  there  a  significant  difference  between  the  vocabulary 
scores  of  the  focus  versus  the  non-focus  group  as 
measured  by  depth  of  knowledge,  automaticity  and 
reading  comprehension? 
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2.  Is  there  a  significant  difference  between  the  groups  with 
and  without  lexical  access  as  measured  by  depth  of 
knowledge,  automaticity,  and  reading  comprehension? 

3.  Is  there  a  significant  interaction  between  group  (focus/ 
non-focus)  and  lexical  access  (yes/no)  on  measures  of 
depth  of  knowledge,  automaticity,  and  reading 
comprehension? 

4.  Is  there  a  significant  difference  between  a  post-test  and 
delayed  post-test  two  weeks  later  on  depth  of  knowledge 
and  automaticity? 

Definition  of  Terms 

The  following  terms  are  defined  for  this  study: 

Acquisition  from  context:  Learning  vocabulary  by  using  cues  from 
the  surrounding  text,  whether  intentionally  or  incidentally. 
Automaticity:  Concerns  the  ease  and  readiness  with  which  a  person 
can  retrieve  his  or  her  lexical  knowledge  from  memory;  in  this  study 
refers  to  the  speed  of  retrieval  as  measured  by  the  student' s 
reaction  time  to  word  identification  tasks  on  the  computer. 

Delayed  testing:  Vocabulary  test  administered  two  weeks  after  the 
initial  test. 

Focus  group:  Refers  to  those  students  who  have  received  instruction 
in  the  definitions  of  the  targeted  words;  consists  of  a  minimum  of  3 
exposures  to  each  targeted  word. 

Foreign  language  (FL)  learning:  Distinguished  from  L2  in  that  it  is 
normally  learned  in  the  absence  of  direct  access  to  L2  culture  and 
learning. 
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Immediate  testing:  Vocabulary  test  to  be  taken  24  hours  after 


reading  the  texts. 

Incidental  learning:  Refers  to  the  acquisition  of  vocabulary  while 
focusing  on  another  task,  such  as  reading;  learning  that  is 
unintentional  in  nature. 

Level  of  word  knowledge:  Refers  to  the  various  degrees  of  word 
knowledge — conceptually  a  continuum  from  zero  knowledge  of  a  word  to 
complete  knowledge  in  every  possible  context,  in  this  study  refers 
to  the  four  level  rating  scale  developed  by  Dale  (1965) . 

Lexical  access:  Refers  to  readers  having  access  to  a  bilingual 
dictionary  while  reading  passages;  in  this  study  both  the  text  and 
dictionary  access  will  be  via  computer. 

Non-focus  group:  Refers  to  those  students  who  received  definitional 
instruction  on  non-targeted  vocabulary. 

Recall  protocol:  A  procedure  in  which  subjects  write  down  (in  their 
LI)  as  much  as  they  can  remember  about  a  text  immediately  after 
reading  it  and  without  referring  back  to  it;  these  recalls  are  then 
scored  based  upon  weightings  assigned  to  various  portions  of  the 
text  (propositions)  contained  in  the  test  according  to  Johnson's 
propositional  analysis  system  (1970) . 

Select-Definition  Test:  Measures  the  ability  of  the  subject  to 
recognize  the  correct  English  definition  for  a  German  word. 
Supply-Definition  Test:  Measures  the  ability  of  the  subject  to 
supply  the  correct  English  definition  or  equivalent  for  a  German 
word. 

Targeted  Vocabulary:  Words  in  the  authentic  texts  which  are  unknown 
to  the  subjects  and  will  be  instructed  to  those  in  the  target  group; 
the  words  will  be  those  that  were  not  instructed  in  their  classes 
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and  confirmed  as  unknown  by  using  Anderson  and  Freebody' s  (1983) 
Yes/No  vocabulary  test. 


Theoretical  Considerations 

Although  incidental  learning  may  hold  the  key  to  large-scale 
vocabulary  development,  there  are  some  significant  problems  in 
relying  solely  upon  this  method.  Some  researchers  have  found  that 
subjects  are  often  unable  to  guess  correct  meanings  from  words  to 
which  they  are  exposed  in  texts  (Kelly,  1990;  Schatz  &  Baldwin, 

1986;  Stein,  1993) .  This  inability  to  derive  correct  meanings  is 
particularly  acute  in  those  learners  with  lower  verbal  skills  (Stahl 
&  Erickson,  1986),  as  well  as  for  low  frequency  words  (Schatz  & 
Baldwin,  1986).  Kelly  (1990)  states  that  when  reading  in  a  FL  "that 
unless  the  context  is  very  constrained,  which  is  a  relatively  rare 
occurrence,  or  unless  there  is  a  relationship  with  a  known  word  .  . 

.  supported  by  context,  there  is  little  chance  of  guessing  the 
correct  meaning"  (p.  203)  .  Incorrect  guesses  at  meaning  can,  in 
turn,  lead  to  further  complications  because  the  students  will 
believe  strongly  that  they  have  derived  the  correct  meaning  and  will 
base  future  interpretations  upon  this  meaning.  It  has  been  shown  in 
comprehension  tasks  that  subjects  are  extremely  reluctant  to  change 
these  meanings  once  they  are  decided  and  will  sometimes  alter  the 
gist  of  the  text  to  make  it  agree  with  these  misunderstood  meanings 
(Bernhardt,  1991) . 

Several  recent  studies  involving  the  use  of  dictionaries 
combined  with  reading  words  in  context  may  shed  light  on  the  process 
of  vocabulary  learning.  This  research  has  all  shown  significant 
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improvements  in  word  knowledge  when  definitional  information  was 
made  available  (Knight,  1992,  1994;  Luppescu  &  Day,  1993) .  These 
same  studies,  along  with  others  (Beck,  McKeown,  &  Omanscn,  1987; 
Gauthier,  1991)  have  also  shown  increases  in  reading  comprehension 
as  a  result  of  vocabulary  instruction.  Another  important  finding 
has  come  from  a  recent  study  by  McKeown  concerning  the  nature  of  the 
definitions  used  for  instruction.  She  states  that  the  reason 
students  often  have  difficulty  learning  from  dictionaries  is  that 
the  included  definitions  are  inappropriate  to  the  task  (McKeown, 
1993) .  Additional  research  has  shown  that  altering  definitions  to 
follow  McKeown' s  guidelines  further  enhances  the  positive  effects  of 
definitional  exposure  (Nist  &  Olejnik,  1995) . 

To  understand  how  vocabulary  learning  takes  place  one  must 
first  look  to  the  current  view  of  word  knowledge.  Researchers  no 
longer  view  a  word  as  either  known  or  unknown,  preferring  the 
concept  of  word  knowledge  as  a  continuum  or  multi-stage  process 
(Beck  6  McKeown,  1991) .  Carey  (1978)  believes  that  words  may 
initially  be  acquired  through  a  process  called  fast-mapping  whereby 
a  central  definition  is  learned  (cited  in  Beck  &  McKeown,  1991)  . 

The  learner's  concept  of  the  word  is  then  gradually  elaborated 
through  slow-mapping,  which  requires  repeated  exposure  to  the  word 
in  a  variety  of  situations.  Along  similar  lines,  Van  Daalen- 
Kapteijns  &  Elshout-Mohr  (1981)  conceptualized  the  process  as  the 
formulation  of  a  rough  meaning  with  empty  slots  for  future,  more 
specific  information. 

Bialystok  (1988)  makes  the  issue  even  more  complex  by  positing 
that  word  knowledge  develops  along  two  separate  and  independent 
continua —  analysis  and  automaticity .  Analysis  is  seen  as  "the 
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level  of  the  awareness  of  the  structure  of  the  linguistic  knowledge 
possessed  by  the  L2  learner"  (Gahren,  1993),  although  it  may  or  may 
not  be  conscious  to  the  learner.  Automaticity  deals  with  the  ease 
with  which  a  person  can  access  their  lexical  knowledge.  One  of  the 
ways  in  which  automaticity  can  be  investigated,  the  method  utilized 
in  this  study,  is  to  identify  the  speed  of  access  by  measuring  the 
response  times  on  identification  and  recall  tasks.  Bialystok  (1988) 
has  posited  that  learners  usually  emphasize  advancement  along  one 
continuum  at  the  expense  of  the  other.  In  particular,  those 
students  in  formal  instruction  settings  normally  give  weight  to 
analysis,  whereas  more  naturalistic  settings  tend  to  boost 
automaticity. 

The  suggestion  that  vocabulary  knowledge  is  a  complex  affair  is 
in  general  agreement  with  cognitive  theories.  Ausabel,  Novak,  and 
Hanesian  (1978)  state  that  the  most  important  aspect  of  learning  new 
material  is  to  establish  a  connection  with  an  existing  anchoring 
idea.  Subsequent  input  will  then  strengthen,  alter,  or  weaken  the 
initial  connections.  This  view  of  concepts  as  a  network  has  grown 
into  the  current  theory  of  Parallel  Distributed  Processing  (PDP) . 

In  this  model  the  brain  is  considered  as  a  complex  network  which 
"learns  a  new  behavior  pattern  by  changing  the  'weight'  of  its 
various. connections  on  the  basis  of  patterns  received  from  input  to 
it."  (Spolsky,  1989,  p.  227)  Although  concepts  are  not  identical  to 
words,  a  similar  process  applies  to  the  learning  of  words. 

Hall  (1992)  posits  that  in  learning  new  L2  vocabulary  learners 
follow  a  "parasitic  strategy  which  ensures  the  simplest  connections 
between  LI  and  L2  representations"  (p.  2),  and  that  they  do  this  by 
seeking  translation  equivalents  in  their  LI.  He  goes  on  to  draw  a 
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distinction  between  knowledge  (concepts)  and  linguistics  (words), 
suggesting  that  meaning  and  thought  are  non-linguistic  (citing 
Jackendoff,  1983),  which,  in  turn  implies  that  they  are  represented 
and  stored  differently  in  the  mental  network.  Therefore,  the 
meaning  of  a  word  is  determined  by  the  concepts  to  which  it  is 
attached.  According  to  this  model,  the  key  to  learning  L2 
vocabulary  is  to  first  establish  the  new  linguistic  forms,  and  then 
make  the  appropriate  connections  to  existing  concepts  (many  of  which 
will  be  in  LI)  .  In  light  of  this  perspective.  Hall  finds  a  useful 
role  for  traditional  vocabulary  instruction  and  believes  that  it 
will  ease  the  cognitive  load  when  the  words  are  subsequently 
encountered  in  text. 

All  of  the  above  conceptualizations  share  a  view  that  word 
knowledge  is  a  complex  affair  and  that  the  learning  of  vocabulary 
is,  likewise,  many-faceted.  Studies  to  date  have  shown  several 
factors  that  seem  to  be  helpful  in  both  incidental  and  direct 
vocabulary  learning.  First,  more  exposures  to  the  new  word  normally 
leads  to  .greater  word  knowledge  (Mezynski,  1983) .  Second,  learning 
is  enhanced  when  the  words  are  provided  with  some  form  of  contextual 
support  which  provides  additional  information  concerning  how  the 
word  should  be  used  (Graves,  1986) .  Lastly,  the  degree  of  analysis, 
or  "depth  of  processing"  plays  a  significant  role  in  how  well  the 
new  vocabulary  is  learned  and  retained  (Williamson,  1989;  Pressley, 
Levin,  &  McDaniel,  1987)  . 

The  factors  that  affect  incidental  learning  can  be  categorized 
as  reader  variables  and  text  variables,  with  the  latter  also 
including  contextual  variables.  First,  readers  vary  "in  their 
ability  to  infer  and  remember  meanings  or  words  encountered  in  text" 
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(Knight,  1992),  those  learners  with  greater  verbal  skills  and 
language  proficiencies  normally  being  able  to  extract  more  words 
from  a  given  text.  Both  cognitive  abilities  and  background 
knowledge  can  impact  the  success  learners  have  in  utilizing  text  and 
contextual  clues.  Also,  certain  characteristics  of  words  such  as 
word  difficulty,  part  of  speech,  and  morphological  transparency 
(Nagy  et  al,  1987),  help  to  determine  the  ease  with  which  the  new 
words  can  be  learned  from  context.  Word  difficulty  relates  to  the 
degree  of  cognitive  work  required  to  incorporate  the  new  word  into 
the  learner's  knowledge  structure,  for  instance  it  is  easier  to 
learn  a  word  referring  to  a  known  concept  than  one  involving  an 
unknown  concept.  Lastly,  contextual  cues  can  play  a  strong  role  in 
either  supporting  or  degrading  the  ability  to  discover  the  meaning 
of  a  new  word  (Sternberg,  1987;  Jenkins  &  Dixon,  1983)  . 

So  how  might  prior  definitional  instruction  assist  the  process 
of  acquisition  from  context?  Certainly  no  claim  can  be  made  that 
exposure  to  a  definition  will  result  in  any  form  of  complete  word 
knowledge,  but  rather  that  students  will  achieve  what  Beck  refers  to 
as  the  "acquainted  level".  Graves  and  Prenn  (1986)  suggest  that 
even  a  brief  encounter  with  a  word  will  leave  some  trace  of  its 
meaning  and  make  students  more  likely  to  fully  grasp  its  meaning 
when  they  come  across  it  again  in  context  (Jenkins,  Stein,  & 

Wysocki,  1984) .  Moreover,  brief  instruction,  provided  immediately 
before  students  read  a  selection  containing  the  word,  "may  be 
sufficient  to  prevent  their  stumbling  over  it  as  they  read"  (Graves 
&  Prenn,  1986,  p.598).  Nation  (1982)  claims  that  some  list  learning 
(definitional  instruction)  has  been  shown  beneficial,  and  that 
presentation  of  words  prior  to  their  appearance  in  texts  can  be 
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effective.  The  addition  of  lexical  access  during  reading  may 
provide  further  assistance  by  way  of  more  exposures  to  the 
definitions,  confirming  or  rejecting  guesses  at  meaning,  and 
reinforcement  of  known  words . 

In  those  situations  in  which  there  a  few  contextual  clues  to 
indicate  meaning,  providing  definitions  may  be  the  only  way  a 
student  can  acquire  certain  words.  Research  has  indicated  that,  at 
least  in  authentic  materials,  it  is  more  often  than  not  the  case 
that  passages  provide  inadequate  contextual  clues  (Kelly,  1990; 
Schatz  &  Baldwin,  1986) .  Vocabulary  instruction  may  also  assist 
incidental  learning  in  a  more  indirect  way  by  igniting  a  greater 
general  interest  in  words  and,  thereby,  improve  acquisition  of  non- 
targeted  words  (Beck,  Perfetti,  &  McKeown,  1982) . 

Assumptions 

The  following  assumptions  were  made  for  this  study: 

1.  All  subjects  will  perform  to  the  best  of  their  ability. 

2.  Rating  of  written  definitions  by  a  panel  of  language 
instructors  is  a  valid  and  reliable  measure  of  depth  of 
word  knowledge. 

3.  Reaction  time  is  a  valid  and  reliable  measure  of  the  level 
of  automaticity  of  the  subject's  word  knowledge. 

4.  Subjects  possess  sufficient  computer  skills  to  accomplish 
the  required  tasks. 
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Limitations 


The  following  are  limitations  of  the  study: 

1.  The  high  verbal  abilities  of  the  subjects  do  not  reflect 
the  general  population. 

2.  Results  may  vary  depending  upon  the  type  of  unknown  words 
selected. 

3.  Results  may  vary  if  the  contexts  in  which  the  words  are 
presented  are  altered. 

4.  The  computerized  instruction  will  be  limited  to  the 
available  technology. 

5.  Generalizations  may  be  limited  to  learners  of  German. 
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CHAPTER  2 


REVIEW  OF  THE  LITERATURE 

Introduction 

This  study  investigates  the  effect  of  direct  instruction  of 
targeted  vocabulary  words  on  subsequent  vocabulary  knowledge  and 
reading  comprehension.  A  review  of  the  SLA  research  shows  that 
vocabulary  has,  until  recently,  been  slighted  by  researchers,  who 
generally  preferred  to  focus  on  other  aspects,  primarily  grammar 
(Gass,  1988;  Hague,  1987) .  Both  LI  and  L2  research  will  be  cited, 
although  the  bulk  of  the  literature  concerns  Ll  studies.  The  review 
of  the  literature  is  organized  as  follows:  (a)  the  roles  of 
incidental  learning  and  direct  instruction  in  vocabulary 
acquisition,  (b)  recent  studies  involving  definitions  and  vocabulary 
learning,  (c)  the  combination  of  definitional  instruction  and 
contextual  exposure,  (d)  vocabulary  knowledge  and  reading 
comprehension,  and  (e)  methods  for  measuring  vocabulary  knowledge 
and  reading  comprehension. 

Incidental  Learning  versus  Direct  Instruction 
The  importance  of  vocabulary  learning  to  language  acquisition, 
particularly  reading  comprehension,  has  generally  been  accepted  by 
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those  involved  in  the  fields  of  LI  and  L2  language  learning  (Stoller 
&  Grabe,  1993;  Chall,  1987)  .  Most  students  of  an  L2  rate  lexis  as 
their  main  concern  (Crow,  1986;  Nunan,  1988) .  Research  on  lexis, 
however,  has  generally  taken  a  back  seat  to  grammar  research.  Much 
of  the  lexical  research  that  was  completed  dealt  with  children 
learning  their  LI  -  how  rapidly  their  vocabulary  knowledge  expanded 
and  how  they  acquired  this  knowledge. 

This  LI  research  indicated  that  the  vocabulary  taught  directly 
in  the  classroom  could  only  account  for  a  small  percentage  of  the 
total  vocabulary  that  children  possessed  (Sternberg,  1987) .  Studies 
were  conducted  in  order  to  determine  the  number  of  words  that  could 
be  learned  incidentally  through  encounters  in  context.  The  results 
of  these  studies  varied  widely,  with  the  study  by  Siragi,  Nation,  & 
Meister  claiming  that  the  learners  were  able  to  learn  68%  of  the 
targeted  words  on  the  one  hand,  and  Nagy  and  Herman  (1987)  finding 
that  their  students  learned  and  average  of  1  in  20  words,  or  roughly 
5%.  The  differences  between  these  studies  helps  to  illuminate  the 
difficulty  in  standardizing  these  studies.  In  the  first  study 
highly  literate  adult  learners  were  exposed  to  artificial  words 
occurring  frequently  in  the  text  (many  18  times  or  more)  .  Nagy  and 
Herman  examined  younger  learners  studying  real  words  occurring  with 
much  lower  frequency.  When  taken  in  total,  the  studies  indicate 
that  the  percentage  of  unknown  words  learned  incidentally  was  rather 
low,  but  given  the  tremendous  number  of  LI  words  to  which  students 
are  exposed  daily,  this  low  rate  is  sufficient  to  provide 
substantial  vocabulary  development.  (Nagy  &  Herman,  &  Andersen, 

1985;  Nagy  &  Herman,  1987)  . 
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As  seen  above,  there  is  a  great  deal  of  variation  in  examining 
learning  words  from  context.  Sternberg  (1987)  has  identified  some 
mediating  variables  responsible  for  this  variation: 


1  Number  of  occurrences  of  the  unknown  word:  multiple 
occurrences  increase  the  number  of  available  cues  and 
can  increase  the  usefulness  of  cues  if  the  reader 
integrates  this  information. 

2  Variability  of  contexts  in  which  multiple  occurrences  of 
the  unknown  word  appears:  variable  contexts  increase  the 
likelihood  that  a  wider  range  of  types  of  cues  will  be 
supplied  and  thus  aid  the  reader. 

3  Importance  of  the  unknown  word  to  understanding  the 
context  in  which  it  is  embedded:  words  perceived  as 
unimportant  may  be  ignored. 

4  Helpfulness  of  the  surrounding  context  in  understanding 
the  meaning  of  the  unknown  word  and  its  proximity. 

5  Density  of  unknown  words:  in  passages  containing  a 
relatively  high  number  of  unfamiliar  words,  context 
provides  less  information  about  the  meaning  of  any  single 
word. 

6  Usefulness  of  prior  knowledge  in  cue  utilization: 
usefulness  will  depend  in  large  part  on  a  given, 
individual's  ability  to  retrieve  information,  to  recognize 
it's  relevance,  and  then  to  apply  it  appropriately. 

(p.  92-94) 


The  factors  listed  above  provide  a  reasonable  synopsis  of  the  major 
variables  shown  to  impact  the  incidental  learning  of  vocabulary,  all 
having  demonstrated  importance  in  incidental  learning. 

The  realization  that  most  words  are  learned  indirectly  does  not 
reduce  the  potential  contribution  of  some  form  of  instruction 
(Sternberg,  1987,  Stahl  &  Fairbanks,  1987).  As  noted  earlier, 
virtually  all  attempts  at  direct  vocabulary  instruction  are,  in 
fact,  successful  to  some  degree  (Beck,  McKeown,  &  Omanson,  1987; 
Kameenui,  Carnine,  &  Freschi,  1982).  In  particular,  mnemonic 
devices  and  the  "keyword"  approach  have  provided  larger  and  more 
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consistent  gains  in  vocabulary  knowledge  than  have  other  methods 
(Pressley,  Levin,  &  McDaniel,  1987) . 

The  different  success  rates  between  studies  are  often  ascribed 
to  several  factors,  primary  among  them  are  the  amount  of  exposure  to 
the  vocabulary,  the  context  in  which  the  exposure  occurs,  as  well  as 
the  'depth  of  processing'  (McKeown  &  Curtis,  1987).  It  has  been 
consistently  shown  that  more  frequent  exposure  to  a  word  in  varied 
contexts  results  in  greater  vocabulary  acquisition,  meaning  that 
studies  which  spend  a  great  deal  of  time  on  each  word  have  an 
inherent  advantage  on  subsequent  word  knowledge  (McKeown,  Beck, 
Omanson,  &  Perfetti,  1983)  .  It  also  appears  that  methods  which  task 
more  mental  capacities  (greater  depth  of  processing)  show  larger 
gains  in  lexis  than  methods  which  fail  to  do  so  (Pressley,  Levin,  & 
McDaniel,  1987;  Nation,  1990) .  The  success  of  the  keyword  method, 
which  requires  learners  to  form  associative  links  between  a  word  and 
its'  definition,  may  be  attributed,  at  least  in  part,  to  this  depth 
of  processing.  The  effect  of  both  of  these  factors,  time  and 
processing  depth,  would  be  supported  by  the  PDP  model  in  that  they 
would  serve  to  strengthen  the  connections  involving  the  word  and  its 
meaning. 

Although  some  researchers  maintain  that  vocabulary  should  be 
acquired  through  context,  the  cited  studies,  taken  in  sum,  lend 
support  to  direct  instruction  as  a  useful  adjunct  for  acquiring 
vocabulary  (McKeown  &  Curtis,  1987;  Huckin,  Haynes,  &  Coady,  1993). 
These  Ll  studies  can  help  inform  research  in  FL  and  L2,  but 
differences  between  these  situations  (Ll  vs  FL/L2)  must  be  used  to 
determine  which  results  are  applicable.  There  are  some  important 
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distinctions  between  LI  and  L2,  as  well  as  differences  between  child 
and  adult  learners. 

Children  learning  their  LI  normally  have  an  existing  functional 
vocabulary  and  a  good  knowledge  of  syntax,  an  advantage  not  shared 
by  L2  learners  (Graves,  1987).  LI  students  are  also  not  influenced 
by  transfer  effects  which  can  impact  L2  learners  (both  negatively 
and  positively) .  L2  learners,  in  particular  mature  learners,  bring 
a  broad  spectrum  of  knowledge  to  the  task  -  conceptually,  in  terms 
of  LI  lexicon,  as  well  as  in  terms  of  cognitive  maturity  (Hague, 
1987).  Finally,  L2  learners,  having  already  learned  their  LI,  may 
have  access  to  well-developed  learning  strategies. 

Given  these  differences,  Stoller  and  Grabe  (1993)  summarize 
the  general  findings  of  LI  studies  that  may  be  applicable  to  L2 : 

-  vocabulary  knowledge  is  the  'cornerstone'  of  literacy 

-  instruction  has  an  impact  on  both  vocabulary  knowledge  and 

reading  comprehension 

-  incidental  learning  may  account  for  a  large  portion  of 
vocabulary  growth  (requires  independent  learning  strategies) 

-  learning  vocabulary  requires  multiple  exposures 

-  learners  must  be  able  to  relate  new  lexical  items  to 
existing  knowledge 

These  statements  suggest  that  both  incidental  learning  and  direct 
instruction  are  capable  of  contributing  to  the  learning  of  LI  and  L2 
vocabulary,  however  further  research  is  needed. 

Situational  differences  also  affect  the  manner  in  which 
vocabulary  is  acquired.  For  most  foreign  language  learners,  the 
kind  of  exposure  that  makes  incidental  learning  of  an  LI  lexicon 
possible  is  normally  not  present  (Allen,  1992;  Hague,  1987).  This 
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suggests  that  "incidental  learning  of  vocabulary  to  any  great  extent 
is  improbable"  (Hague,  1987,  p.  220),  particularly  in  the  early 
stages  of  language  learning.  Kelly  (1990)  cites  some  potential 
problems  of  FL  students  'guessing'  the  meanings  of  words  by  their 
context.  Other  researcher  also  find  this  'guessing'  problematic, 
asserting  that  students  often  lack  sufficient  knowledge  to  access 
the  correct  meanings  of  words  and,  as  a  result,  often  mis-identify 
words  (Haynes,  1993;  Holmes  &  Ramos,  1993).  It  would  appear  that 
one  role  of  instruction  may  be  to  provide  the  necessary  bottom-up 
knowledge  required  for  correct  'guessing'  of  word  meanings,  or 
incidental  learning.  None  of  the  L2  studies  suggests  that  there  is 
no  role  for  incidental  learning;  almost  all  see  at  least  some  role 
for  direct  instruction  (McKeown  &  Curtis,  1987;  Huckin,  Haynes,  & 
Coady,  1993) . 

There  has  been  an  increase  in  the  overall  amount  of  L2 
vocabulary  research,  but  much  of  it  has  been  concerned  with  problems 
of  contextual  guessing,  learner  strategies,  and  impact  on  reading 
comprehension.  Two  of  the  studies  that  have  dealt  with 
instructional  approaches  are  those  of  Brown  (1993)  and  Coady  (1993) . 
Brown  found  that  the  following  factors  affect  vocabulary 
acquisition:  (a)  general  frequency,  (b)  word  saliency  (importance) 
in  the  specific  context,  and  (c)  prior  knowledge  of  the  concept 
involved.  The  study  by  Coady  (1993)  investigated  whether  "computer- 
assisted  instruction  in  high  frequency  vocabulary  items  will 
increase  the  amount  of  sight  vocabulary"  (p.  220)  and,  in  turn, 
reading  comprehension.  The  results  of  his  study  showed 
statistically  significant  improvement  in  vocabulary  knowledge  and 
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comprehension,  as  well  as  a  positive  response  on  the  part  of  the 
students  for  the  computerized  instruction  program. 

In  summary,  the  emphasis  on  incidental  learning,  predicated  on 
the  early  LI  research,  has  been  adapted  to  include  a  role  for 
certain  degrees  of  vocabulary  instruction.  In  L2  research,  "a  more 
viable  approach  to  second-language  reading  and  word-guessing  appears 
to  be  one  in  which  learners  employ  both  top-down  and  bottom-up 
processing  in  complementary  fashion"  (Huckin  &  Haynes,  1993,  p. 

291)  .  Vocabulary  instruction,  in  other  words,  in  addition  to 
providing  direct  word  knowledge,  enables  learners  to  acquire  words 
from  context. 

Recent  Studies  Involving  Definitions  and  Vocabulary  Learning 

The  number  of  studies  concerning  dictionary  use  and  vocabulary 
learning  has  been  quite  small  (Nist  &  Olejnik,  1995),  and  much  of 
this  research  does  not  support  the  use  of  dictionary  to  increase 
vocabulary  (Crist,  1981;  Eeds  &  Cockrum,  1985)  .  The  lack  of 
research  becomes  even  more  pronounced  when  examining  L2  research, 
but  recent  studies  by  Knight  (1993)  and  Luppescu  &  Day  (1993)  have 
begun  to  address  these  issues.  More  recently  Nist  &  Olejnik  (1995) 
have  investigated  the  interaction  of  context  and  access  to 
definitions  on  levels  of  word  knowledge.  Although  this  latter  study 
is  in  LI,  many  aspects  of  the  study  can  inform  L2  research.  In 
particular,  the  emphasis  on  the  adequacy  and  appropriateness  of  the 
definitions,  as  specified  by  McKeown  (1993),  should  apply  equally  to 
foreign  language  learning. 

The  Knight  study  investigated  whether  dictionary  access  and 
verbal  ability  would  impact  students'  incidental  vocabulary  learning 
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from  context.  The  subjects  for  her  study  were  105  students  enrolled 
in  Spanish  201  at  Central  Michigan  University.  These  students  were 
then  randomly  selected  into  either  the  group  with  dictionary  access 
or  the  group  without  access.  These  students  were  then  exposed  to 
two  authentic  Spanish  articles  that  had  been  selected  as  appropriate 
by  a  team  of  Spanish  instructors.  Twelve  targeted  words  were  chosen 
from  each  text  and  verified  as  unknown  by  the  students.  Following 
the  reading  of  the  texts,  the  students  were  given  vocabulary  and 
comprehension  tests.  Specifically,  vocabulary  knowledge  was  tested 
via  a  supply-definition  test  and  a  select-definition  test,  and 
reading  comprehension  was  tested  by  an  immediate  recall  protocol.  A 
second  test  of  vocabulary  knowledge  was  given  two  weeks  after  the 
first  test  to  examine  the  delayed  effects.  The  results  were 
subjected  to  analysis  by  ANOVA. 

In  sum,  the  study  found  that  students  can  learn  new  words 
incidentally,  but  that  "high  verbal  ability  students  learn  more 
words  than  low  ability  students,  and  students  who  use  a  dictionary 
learn  more  than  those  who  do  not"  (Knight,  1994,  p.  292) .  The 
supply-definition  tests  showed  that  those  students  without 
dictionary  access  learned  five  to  seven  percent  of  the  targeted 
unknown  words,  whereas  the  students  with  access  to  definitions 
learned  between  19  to  21  percent  of  the  unknown  words.  When  tested 
with  select-definition  tasks  the  scores  more  than  doubled,  but  those 
with  dictionary  access  maintained  a  significant  advantage.  Although 
all  groups  benefited  from  dictionary  access,  it  provided  a  special 
advantage  to  the  low  verbal  ability  group.  In  terms  of  reading 
comprehension,  the  means  for  those  with  dictionary  access  were 
consistently  higher  than  for  those  without  access,  though  a 
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significant  difference  was  noted  only  for  the  low  verbal  ability 
group.  The  study  also  found  that  reading  time  increased  40-45 
percent  for  the  students  with  dictionary  access,  but  that  "the 
amount  of  vocabulary  learned  increases  in  greater  proportion7' 

(Knight,  1994,  p.  294) .  Overall,  the  study  shows  that  access  to 
definitions  provides  benefits  in  both  vocabulary  knowledge  and 
comprehension,  especially  for  learners  with  lower  verbal  abilities. 

Another  study  which  examines  the  use  of  dictionaries  during 
reading  and  subsequent  vocabulary  knowledge  is  that  of  Luppescu  and 
Day  (1993)  .  They  believed  there  would  be  no  significant  difference 
between  the  group  with  dictionaries  and  those  without  dictionaries 
based  upon  Krashen' s  implication  that  vocabulary  acquired  naturally 
will  be  more  persistent  than  that  which  is  learned  via  definitions. 

A  second  focus  of  the  study  was  the  amount  of  time  both  groups  would 
require  to  read  the  assigned  story. 

The  study  involved  293  first-  and  second-year  Japanese  students 
studying  English  as  a  foreign  language.  They  were  randomly  assigned 
to  one  of  the  categories  (dictionary/no  dictionary)  and  instructed 
to  read  a  short  story  containing  17  targeted  words  that  the 
participants  in  the  study  either  did  not  know  or  found  difficult. 

The  original  text  of  the  story  had  been  altered  to  increase  the 
frequency  and  the  contextual  support  of  the  targeted  words.  After 
reading  the  story,  the  subjects  took  a  select-definition  test  to 
measure  their  vocabulary  knowledge. 

An  analysis  of  the  results  indicates  that  the  means  of  the 
vocabulary  scores  for  those  students  who  used  dictionaries  was  50% 
higher  than  for  students  without  dictionaries.  This  was  in  direct 
contradiction  to  the  expectation  of  the  investigators.  Consistent 
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with  the  study  by  Knight,  this  study  also  found  that  reading  time 
for  the  dictionary  users  was  twice  that  of  group  with  no  access. 

One  interesting  finding  was  that  the  results  varied  with  item 
difficulty,  leading  the  researchers  to  conclude  that  use  of  a 
dictionary  may  sometimes  be  'misleading  or  confusing' .  This  concept 
that  the  definitions  themselves  may  be  part  of  the  problem  is  dealt 
with  in  a  paper  witten  by  McKeown  (1993)  . 

McKeown  takes  the  view  that  definitions  one  finds  in 
traditional  dictionaries  are  not  necessarily  set  in  a  form 
appropriate  for  learners.  Definitions  are  normally  written  in 
fragmented  language  and  often  use  multiple  listings  in  an  attempt  to 
include  all  possible  meanings.  McKeown  (1993)  believes  these 
factors  can  confuse  learners  and  make  finding  the  correct  meaning 
for  a  word  a  complex  process.  This  process,  as  described  by  Lupescu 
and  Day  (1993),  "may  entail  looking  for  a  suitable  headword, 
comprehending  the  entry,  locating  the  appropriate  part  of  the 
definition,  and  connecting  with  the  right  sense  of  context"  (p. 

274).  McKeown  prefers  to  view  definitions  as  an  "initiating  event", 
"unlikely  to  promote  a  complete  understanding  of  a  word"  (p.17) 
which  requires  repeated  exposures,  but  providing  a  summary  meaning 
which  learners  can  readily  grasp  and  from  which  they  can  expand 
their  understanding  of  the  word. 

She  provides  the  following  principles  for  writing  definitions: 
(a)  identify  the  essence  of  a  word  and  its  role  in  language,  (b) 
pinpoint  the  word's  characteristic  use,  (c)  make  the  word  accessible 
to  the  learner,  and  (d)  arrange  for  attention  to  the  whole 
definition  rather  that  just  a  fragment.  The  following  example 
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illustrates  the  traditional  and  revised  definitions  for  the  word 
covert . 

Traditional:  kept  from  sight;  secret;  hidden 
Revised:  describes  something  that  is  done  in  a  hidden  or 

secret  way 

Her  study  indicated  greater  learning  by  students  provided  with 
revised  definitions  and,  in  general,  indicates  that  definitions  can 
aid  vocabulary  development  "by  initiating  the  process  of  knowing  a 
word  or  by  providing  a  coherent  summary  of  meaning  that  can  be  used 
as  a  reference"  (McKeown,  1993,  p.  29). 

Nist  and  Olejnik  (1995)  make  use  of  McKeown' s  definition 
research  in  their  study  investigating  the  interaction  between 
context  and  definitions,  a  subject  not  previously  investigated.  In 
this  LI  study  the  researchers  randomly  assigned  186  college  freshmen 
to  one  of  four  groups  encompassing  all  the  possible  combinations  of 
weak  and  strong  context  and  inadequate  (traditional)  and  adequate 
(revised  according  to  McKeown' s  guidelines)  definitions.  The 
subjects  were  given  twenty  minutes  to  learn  the  vocabulary  from  the 
materials  they  were  given.  "Each  word  was  presented,  first,  in 
either  a  strong  or  weak  context,  immediately  followed  by  an  adequate 
or  inadequate  definition"  (Nist  &  Olejnik,  1995,  p.  181) .  The 
students  were  then  tested  on  their  knowledge  with  two  different 
forms  of  multiple  choice,  sentence  generation,  and  fill  in  the 
blank. 

The  major  finding  of  the  study  was  that  providing  adequate 
definitions  made  a  significant  difference  in  all  four  of  the 
vocabulary  tests.  Those  students  with  adequate  definitions  achieved 
60%  accuracy  on  the  sentence  production,  and  80-90%  on  the  remaining 
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tasks.  The  effects  for  context  and  the  interaction  between 
definition  and  context  were  non-significant  across  all  tests.  The 
failure  of  context  to  be  significant  may  be  related  to  the  way  the 
words  were  presented,  one  by  one  rather  than  incorporated  into  a 
coherent  text,  leading  the  students  to  view  learning  the  definition 
as  the  primary  task.  The  researchers  also  posit  that  using 
definitions  might  be  a  preference  in  older  learners.  Overall,  the 
study  supports  the  use  of  definitions  in  vocabulary  learning, 
particularly  definitions  revised  to  be  appropriate  for  learners. 

These  recent  studies  have  all  shown  that  providing  access  to 
definitions  has  resulted  in  significant  gains  in  subsequent 
vocabulary  knowledge.  Reading  times  are  considerably  increased,  in 
the  region  of  40-50%,  but  there  was  an  even  greater  increase  in  word 
knowledge.  Knight's  (1992)  finding  that  there  was  an  increase  in 
reading  comprehension  is  also  in  agreement  with  past  studies. 

Definitional  Instruction  and  Contextual  Exposure 

The  possibility  that  definitional  instruction  will  benefit 
subsequent  contextual  exposure  is  based  upon  many  of  the  issues 
already  discussed.  If  one  accepts  the  notion  that  word  knowledge 
develops  from  a  rough  representation  and  slowly  expands  and  evolves 
by  developing  and  strengthening  mental  connections  (van  Daalen- 
Kapteijns  &  Elshout-Mohr,  1981;  Hall,  1992) ,  then  prior  instruction 
provides  an  "intiating  event"  (McKeown,  1993)  resulting  in  an 
general  acquaintance  with  the  word.  This  familiarity  may  prevent 
stumbling  over  these  words  when  seeing  them  in  text  and  allow  for 
greater  concentration  on  other  aspects  of  the  passage  (Graves  & 
Prenn,  1986).  The  instructed  words  may  also  provide  contextual 
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clues  for  unknown,  or  lesser  known,  vocabulary.  In  addition,  the 
possession  of  a  rough  meaning  can  prevent  some  of  the  erroneous 
guesses  that  might  otherwise  occur  (Kelly,  1990) .  These  false 
meanings,  if  they  are  located  in  a  critical  segment  of  the  passage, 
can  adversely  impact  the  reader's  understanding  of  the  overall  text 
(Bernhardt,  1991)  .  Admittedly,  the  representation  for  a  word  that 
most  students  will  possess  after  brief  instruction  is  vague,  but 
that  crude  conception  may  be  adequate  to  provide  a  foundation  from 
which  to  make  informed  elaborations,  as  well  as  preventing  some 
inappropriate  guesses  at  meaning. 

Many  of  the  justifications  above  can  be  applied  equally  to  the 
success  of  the  lexical  access  studies  cited.  What  has  not  been 
addressed,  however,  and  of  interest  in  the  current  study,  is  the 
possible  interaction  between  prior  instruction  and  lexical  access 
during  reading. 

There  are  several  reasons  to  believe  that  there  may  be  a 
significant  effect  when  prior  instruction  and  lexical  access  are 
combined.  First,  there  is  the  straightforward  matter  of  frequency  - 
-  the  more  often  that  students  are  exposed  to  a  word  the  more  likely 
they  are  to  learn  and  retain  its  meaning.  The  strong  effect  for 
frequency  is  one  of  the  most  consistent  findings  in  vocabulary 
research  (Beck  et  al . ,  1983:  Graves,  1987;  Brown,  1993).  Second, 
there  is  the  benefit  of  subsequent  reinforcement  of  the  definitions, 
cited  as  beneficial  by  Pimsleur  (1967)  as  well  as  Ausabel,  Novak,  & 
Hanesia  (1978)  .  The  latter  group  has  proposed  an  "immunizing" 
effect  that  occurs  because  "trying  to  remember  makes  the  learner 
aware  of  relevant  related  concepts  in  cognitive  structure"  (p.  328), 
and  the  learner  is  therefore  in  a  better  position  to  learn  from 
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subsequent  encounters.  Lastly,  the  availability  of  definitions 
during  reading  also  allows  the  reader  to  more  accurately  confirm  or 
reject  guessed  meanings.  It  is  the  case  that  many  students, 
particularly  adults,  like  to  confirm  words,  even  those  they  feel 
secure  about.  This  confidence  in  the  base  definition  may  then  allow 
them  to  investigate  the  text  for  possible  differentiation  or 
subtleties  in  the  word's  meaning. 

Many  researchers  have  argued  for  a  mixed  approach  (Beck  et  al., 
1983;  Huckin  &  Haynes,  1993;  Mezynski,  1983;  Stoller  &  Grabe,  1993), 
and  this  combination  of  definitions  in  isolation  followed  by 
definitions  in  a  meaningful  context  is  in  accordance  with  the 
suggestion  that  words  should  be  seen  frequently  under  varied 
conditions.  Although  Nist  and  Olejnik  (1995)  failed  to  find  any 
interaction  between  definitions  and  context,  their  study  was 
different  in  two  important  aspects — the  artificial  manner  in  which 
the  words  were  presented  and  the  limited  (one  sentence)  context. 
Given  prior  instruction  to  establish  a  rough  meaning,  the  evidence 
suggests  that  there  may  well  be  a  symbiotic  effect  when  lexical 
access  is  present  during  the  reading  task. 

Vocabulary  Knowledge  and  Reading  Comprehension 
The  research  to  date  has  established  a  strong  and  consistent 
relationship  between  vocabulary  knowledge  and  reading  comprehension, 
both  in  LI  and  L2  (Curtis,  1987;  Huckin,  Haynes,  &  Coady,  1993)  . 
These  findings  are  problematic  from  the  standpoint  that  they  do  not 
determine  cause  and  effect,  or  stated  another  way,  it  is  unknown 
whether  vocabulary  knowledge  leads  to  good  reading,  or  if  good 
readers  learn  more  vocabulary.  Some  researchers  forward  a  two-sided 
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approach,  suggesting  that  the  two  factors  nurture  each  other. 
Whatever  the  actual  nature  of  the  link,  this  relationship  is  a 
primary  motivation  for  much  of  what  is  done  in  vocabulary 
instruction  (Nagy  &  Herman,  1987)  . 

Mezynski  (1983)  posited  four  positions  (modified  from  Anderson 
and  Freebody)  to  attempt  to  explain  the  link  between  vocabulary 
knowledge  and  reading  comprehension.  The  four  positions  include  a 
belief  in  an  innate  mental  mechanism  (Aptitude) ,  the  knowledge  of 
words  as  a  prerequisite  to  comprehension  (Instrumental) ,  that  word 
knowledge  represents  the  general  knowledge  needed  for  comprehension 
(Knowledge),  an  that  automaticity  of  word  knowledge  aids  aids 
comprehension  (Access).  Hague  (1987)  believed  that  a  combination  of 
these  theories  would  be  more  useful  in  illuminating  the 
relationship.  The  instructional  design  hypothesis  (Kameenui,  Dixon, 
&  Carnine,  1987)  posits  that  all  four  of  the  above  hypotheses  are 
combined  to  form  a  balanced,  multi-variable  explanation  of  the 
relationship  between  lexical  knowledge  and  reading  comprehension. 

It  must  be  recognized  that  lexical  knowledge  is  only  one  factor 
in  determining  reading  comprehension.  Bernhardt  (1991)  in  her 
Constructivist  Model  alludes  to  six  factors  (including  word 
recognition)  that  impact  on  reading  comprehension.  In  light  of 
these  many  facets  the  question  remains — can  instruction  of 
vocabulary  improve  reading  comprehension?  Stahl  and  Fairbanks 
(1987)  found  vocabulary  instruction  provided  small  but  significant 
gains  in  comprehension  of  general  passages,  and  that  the  gains  were 
even  more  positive  for  passages  containing  the  instructed  words 
(Nagy  &  Herman,  1987).  Though  most  of  the  instruction  that  resulted 
in  gains  in  reading  comprehension  can  be  characterized  by  multiple 
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exposures  to  a  word  in  rich  and  varied  meaningful  contexts  (McKeown 
et  al.,  1983)  ,  it  is  very  possible  that  less  intensive  methods  can 
also  be  effective  (Nagy  &  Herman,  1987).  The  assistance  provided  by 
this  type  of  preparatory  instruction  should  be  amplified  when  the 
targeted  words  are  contained  within  the  passage  to  be  comprehended. 
Though  the  jury  is  still  out  on  the  usefulness  of  vocabulary 
instruction  for  reading  comprehension,  there  is  sufficient  evidence 
to  continue  investigating  the  possibilities. 

Measurement  of  Vocabulary  Knowledge  and  Reading  Comprehension 
A  central  question  when  investigating  word  knowledge  is  -  "What 
does  it  mean  to  know  a  word?"  Richards  (1976)  elaborated  on  this 
subject  suggesting  that  word  knowledge  extended  far  beyond  a 
definitional  basis,  extending  to  items  such  as  syntactic  behavior, 
underlying  forms,  the  network  of  associations  the  word  might  have, 
among  others  (Coady,  1993)  .  Researchers  have  posited  various  scales 
for  determining  word  knowledge,  a  prevalent  example  being  that  of 
Drum  and  Konopak  (1987)  in  which  they  present  six  levels  of  word 
knowledge: 

1  knows  a  word  meaning  aurally  but  not  in  written  form 

2  knows  a  word  meaning  but  cannot  express  it 

3  knows  a  meaning  but  not  the  word  for  it 

4  knows  the  partial  meaning  of  a  word 

5  knows  a  different  meaning  for  a  word 

6  knows  neither  the  word  nor  the  concept 

Curtis  (1987)  reduces  the  number  of  levels  to  four,  and  Graves 
(1987)  takes  an  alternate  view  of  word  knowledge  based  on  the  type 
of  learning  task.  Graves  also  makes  an  important  distinction 
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between  words  learned  for  receptive  purposes  and  those  learned  for 
productive  purposes. 

This  study  concerns  itself  primarily  with  the  receptive 
vocabulary,  which  is  thought  to  be  much  larger  than  a  person's 
productive  vocabulary.  Gahren  (1993,  citing  Nation,  1990)  states 
that  "lexical  items  learned  for  receptive  use  allow  the  learner  to: 

(a)  distinguish  the  learned  word  from  other  words  of  similar  form, 

(b)  have  an  expectation  of  a  grammatical  pattern  of  the  word,  (c) 
recognize  some  of  the  collocations  of  the  word,  (d)  know  how  often 
the  word  is  used,  and  (e)  recall  the  meaning  of  the  word  when 
needed"  (  p.  18)  .  The  main  point  is  that  there  are  many  aspects  to 
describing  word  knowledge,  and,  therefore,  that  it  defies  simple 
categorization  as  known  or  unknown. 

Despite  this  obvious  complexity,  many  researchers,  driven 
largely  by  practical  concerns,  have  tested  word  knowledge  as  if  it 
could,  in  fact,  be  viewed  as  a  dichotomy.  Commonly  utilized 
vocabulary  measurement  instruments  include  multiple  choice, 
constructed  answer,  checklist  formats,  and  matching  (Anderson  & 
Freebody,  1981) .  Multiple  choice  measures  are  the  most  popular,  and 
are  capable  of  measuring  partial  knowledge  if  test  items  and 
distracters  are  well  constructed  (Curtis,  1987;  Nagy  &  Herman, 

1987) .  Checklists  that  require  the  subjects  to  indicate  yes/no  as 
to  whether  they  knew  a  word  have  also  been  utilized.  Despite  a 
tendency  for  the  subjects  to  answer  yes  to  more  words  than  they 
actually  knew,  the  results  of  Drum  and  Konopak,  "and  those  of  Nagy, 
Herman,  and  Anderson  do  corroborate  the  general  accuracy  of  the 
Yes/No  approach"  (Drum  &  Konopak,  1987,  p.  81)  .  Constructed 
response,  which  requires  the  subject  to  provide  definitions  for  the 
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targeted  words,  holds  a  great  deal  of  potential  for  demonstrating 
various  levels  of  word  knowledge  (Nation,  1990).  The  success  of 
this  method  depends,  to  a  large  degree,  on  the  abilities  of  those 
rating  the  responses  (Anderson  &  Freebody,  1981) . 

A  qualitative  scale  developed  by  Dale  (1965)  and  utilized  by 
Curtis  (1987)  will  be  used  to  determine  depth  of  word  knowledge  in 
this  study.  The  scale  consists  of  the  following  stages: 

Stage  1:  "I  never  saw  it  before." 

Stage  2:  "I've  heard  of  it,  but  I  don't  know  what  it  means." 

Stage  3:  "It  has  something  to  do  with."  (Partial  knowledge) 

Stage  4:  "I  know  it."  (Correct  response) 

The  student  is  first  asked  to  select  whether  a  word  is  unknown, 
recognized  but  the  meaning  unknown,  or  if  the  word  is  known 
(includes  partial  knowledge) .  In  every  instance  the  student  is 
asked  to  provide  a  meaning,  guessing  if  necessary,  which  is  later 
analyzed  by  raters  to  determine  if  the  given  meaning  is  unknown, 
partially  correct,  or  correct. 

Another  aspect  of  vocabulary  knowledge,  largely  unaddressed,  is 
that  of  automaticity,  or  ease  of  lexical  access.  Bialystok  (1988) 
describes  a  model  that  has  two  independent  dimensions  of  analysis 
and  automaticity.  Analysis  is  the  equivalent  to  depth  of  knowledge 
as  used  in  this  study.  Automaticity  concerns  itself  with  the  speed 
and  ease  that  one  can  apply  their  vocabulary  knowledge  to  a  given 
task. 

Gahren  (1993)  investigated  the  issue  of  automaticity  in  his 
study  which  compares  natural  and  instructed  acquisition  of 
vocabulary.  His  study  of  29  advanced  learners  of  French  at  the  Air 
Force  Academy  included  measures  of  vocabulary  identification  and 
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recall  times  as  indications  of  speed  of  access  to  the  learners 
internal  lexicon.  A  computerized  checklist  was  used  to  record  the 
time  between  the  presentation  of  a  word  and  the  subject's  decision 
on  whether  he  or  she  knew  the  word.  His  study  indicated  gains  in 
automaticity  (as  well  as  for  depth  of  knowledge)  over  the  six  month 
length  of  the  study. 

In  terms  of  reading  comprehension,  traditional  measures  have 
included  cloze,  multiple-choice,  and  direct  content  questions. 
Bernhardt  (1991)  discusses  the  disadvantages  of  these  techniques  and 
outlines  the  facets  of  an  acceptable  measure  of  reading 
comprehension:  (a)  the  measure  must  acknowledge  the  status  of  the 

reader's  knowledge  base,  (b)  the  assessment  mechanism  must  be 
integrative  in  nature,  and  (c)  should  provide  process  information  in 
addition  to  quantifiable  data.  The  recall  protocol  procedure  is 
proposed  as  an  answer  these  challenges.  Subjects  are  allowed  to 
examine  the  targeted  passages  as  long  as  they  desire,  and  then  write 
as  much  as  they  can  remember  in  their  LI.  Due  to  its  constructive 
open-ended  nature,  "generating  recall  data  does  not  influence  a 
reader's  understanding  of  the  text"  (Bernhardt,  1991,  p.  200)  . 
Allowing  subjects  to  answer  in  their  LI  removes  the  confounding 
effect  of  L2  production  and,  therefore,  gives  a  clearer  picture  of 
what  has  been  comprehended.  The  obvious  disadvantage  of  the 
protocol  is  that  there  is  a  great  deal  of  subjectivity  in  assigning 
weights  to  the  various  propositions,  or  portions  of  text.  The 
protocols  used  in  this  study  have  been  part  of  a  recent  study 
conducted  by  Heinz  (1993)  at  the  Air  Force  Academy  in  which  he 
evaluated  the  computerized  scoring  of  recall  protocols. 


35 


Heinz  administered  three  texts  to  cadets  studying  German  and 
asked  them  to  provide  recall  protocols  for  each  of  the  texts.  The 
texts  were  then  hand-scored  by  three  German  professors  and  submitted 
for  computerized  scoring  on  a  program  written  by  Professor  Heinz. 

The  results  indicated  a  strong  correlation  (approximately  .90) 
between  hand  and  computerized  scoring.  The  computerized  method  also 
proved  100%  reliable  assigning  the  exact  same  scores  to  all  of  the 
protocols  on  a  second  pass.  These  results  are  significant  because 
hand-grading  protocol  is  such  a  lengthy  procedure  that  it  is,  to  a 
large  extent,  impractical.  The  study  moves  protocols  one  step 
closer  to  practical  reality.  The  grading  sheets  and  the 
computerized  program  were  used  in  the  current  study. 
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CHAPTER  3 


PROCEDURES 


Population  and  Sample 

The  subjects  were  the  cadets  enrolled  in  the  beginning  and 
intermediate  phases  of  German  language  study  at  the  Air  Force 
Academy  located  in  Colorado  Springs,  Colorado.  Cadets  gain  entrance 
to  the  Academy  through  a  highly  competitive  process  that  examines 
their  academic,  physical,  and  leadership  qualities.  Most  of  the 
cadets  were  among  the  top  10%  of  high  school  students  and  display 
high  levels  of  general  intelligence  and  LI  literacy,  as  evidenced  by 
standardized  tests.  The  4400  cadets,  generally  nominated  by 
congressional  district,  reflect  the  population  densities  found  in 
the  United  States.  Though  perhaps  not  a  perfect  mirror  of  the 
society  at  large,  an  active  minority  recruiting  program  ensures  a 
representation  of  the  diversity  found  in  our  society.  The  cadets, 
after  enduring  an  intense  four  years  of  academic,  physical,  and 
military  training  earn  a  Bachelors  degree  and  are  commissioned  as 
Lieutenants  in  the  Air  Force. 

Except  for  the  roughly  15%  of  new  cadets  who  demonstrate 
sufficient  proficiency  to  validate  out  of  the  language  requirement, 
there  is  a  mandatory  one  year  course  of  foreign  language  study  for 
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all  freshman.  The  cadets  submit  their  selections  from  the  seven 
available  choices  of  languages  and  an  effort  is  made  to  maximize  the 
fit  between  slots  available  in  the  various  languages  and  the  desires 
of  the  students.  German  132  and  German  142  are  the  beginning  and 
intermediate  levels  for  the  freshman  German  classes.  The  total 
number  enrolled  in  these  classes  has  traditionally  been  between  100 
and  120  cadets.  They  receive  one  hour  of  language  instruction  daily 
throughout  their  freshman  year  for  a  total  of  roughly  160  hours  of 
instruction.  Most  cadets  discontinue  their  language  study  once  the 
requirement  is  'fulfilled,  although  some  continue  on  to  earn  a  minor 
in  a  language  or  decide  to  major  in  Foreign  Area  Studies. 

Scores  for  sub-skills  in  the  area  of  verbal  ability  such  as 
those  provided  by  the  ACT  or  SAT  were  acquired  from  the  Registrar. 
Specifically,  overall  verbal  skill  was  available  for  all  the 
participants,  and  the  sub-skills  of  word  knowledge  and  reading 
comprehension  were  obtained  for  roughly  one  third  of  the  subjects. 

Resources 

The  Air  Force  Academy  provided  all  the  resources  necessary  — 
subjects,  the  computerized  language  lab,  technical  expertise,  and 
fellow  German  instructors  to  assist  in  various  selections  and 
ratings,  as  well  as  advice  on  research  and  statistics.  There  were 
also  computer  personnel  who  assisted  with  the  programming  skills 
needed  to  develop  the  instruction  program  and  the  vocabulary  test. 

Research  Design 

The  study  utilized  a  Mulitvariate  Factorial  Analysis  of 
Covariance  design  to  answer  questions  about  the  effect  of  direct 
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instruction  of  targeted  vocabulary  on  subsequent  vocabulary 
knowledge  and  comprehension.  The  two  independent  variables  are 
instruction  of  the  targeted  vocabulary  (yes  or  no),  and  on-line 
dictionary  access  during  the  protocols.  The  dependent  variables  are 
the  vocabulary  scores  for  automaticity  and  depth  of  knowledge,  and 
the  recall  protocol  scores.  The  covariant  was  the  subjects'  verbal 
scores  from  the  SAT  and  ACT.  Lastly,  the  student  responses  to 
questions  regarding  their  perceptions  and  attitudes  about  the 
computerized  programs,  as  well  as  to  the  study  in  general,  were 
tallied  and  analyzed  in  hopes  of  providing  some  qualitative  insight 
to  the  data. 


No  Instruction  on 

Instruction  on 

Targeted  Words 

Targeted  Words 

No  Lexical  Access 

Group  0 

Group  1 

Lexical  Access 

Group  2 

Group  3 

Table  1 

Data  Matrix  for  Two-Factor  Design 
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Instruction  of  the  Targeted  Vocabulary 

This  dichotomous  variable  addressed  whether  or  not  the  subject 
has  received  definitional  instruction  on  the  18  targeted  vocabulary 
selected  from  three  texts.  One  half  of  the  German  132  and  German 
142  cadets  were  randomly  chosen  to  receive  prior  instruction  in 
these  words,  and  the  remainder  received  instruction  in  words  that 
were  not  targeted.  The  instruction  consisted  of  a  minimum  of  three 
exposures  to  the  definition  of  each  word  —  two  in  which  the  words 
are  shown  with  their  definitions,  and  then  a  test  situation  to  see 
if  the  subjects  could  select  the  correct  definition  from  a  list  of 
alternatives.  Once  the  test  was  complete  there  was  an  opportunity 
for  the  cadets  to  go  back  and  review  any  of  the  definitions . 

The  entire  instructional  program  was  delivered  by  computer. 

The  individual  cadets  set  their  own  pace  in  reviewing  the  words,  the 
only  restriction  being  that  there  was  a  maximum  of  45  minutes  to 
review  all  eighteen  words.  The  results  of  other  recent  computerized 
vocabulary  studies  (Wheatley,  Muller,  &  Miller,  1993)  suggested  that 
this  would  be  more  than  adequate  and  should  allow  for  plenty  of  time 
for  additional  review. 

Dictionary  Access 

This  variable  refers  to  whether  or  not  the  subjects  had  access 
to  on-line  definitions  while  they  were  reading  the  texts.  Once 
again,  the  placement  of  the  subjects  into  one  of  these  categories 
was  completely  random.  Those  cadets  who  had  this  access  were  able 
to  view  the  definition  of  any  word  in  the  text  by  using  the  computer 
to  access  the  chosen  word  from  an  on-screen  list.  The  definition 
then  appeared  in  a  window.  Once  the  cadet  was  satisfied,  he  or  she 
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was  able  to  close  the  definition  window  and  move  on  through  the 
text.  The  number  of  times  each  definition  was  accessed  was 
recorded. 


Instrumentation 

Word  Identification  Task 

Each  subject  was  presented  with  a  word  on  the  computer  screen 
and  asked  to  indicate  their  knowledge  about  a  word  by  clicking  on 
the  appropriate  choice  on  the  computer  screen.  The  three  choices 
were:  (a)  unknown,  (b)  recognized  but  the  meaning  is  unknown,  and 
(c)  known  (includes  incomplete  or  partial  knowledge.  The  time  lapse 
between  the  initial  presentation  of  the  word  and  the  response  was 
recorded.  This  time  lapse  was  used  as  a  measure  of  an  aspect  of 
automaticity  —  speed  of  retrieval. 

Word  Recall  Task 

Subjects  were  presented  with  the  18  targeted  German  words  and 
asked  to  supply  a  definition  for  each  word.  The  definitions  were 
written  in  English  to  ensure  that  the  cadets  were  able  to  fully 
articulate  what  they  thought  the  words  meant.  Three  German 
professors  acted  as  independent  raters  of  the  supplied  definitions. 
The  following  grading  criteria  were  used:  (a)  no  points  for  answer 

with  no  correct  knowledge,  (b)  one  point  when  the  subject  indicates 
recognition  of  the  word  with  no  further  knowledge,  (c)  two  points 
for  an  answer  indicating  partial  knowledge,  and  (d)  three  points  for 
an  answer  indicating  a  correct  meaning. 


41 


Recall  Protocols 


Recall  protocols  were  used  to  assess  the  students' 
comprehension  of  the  three  texts,  which  the  students  were  asked  to 
read.  The  protocol  procedure,  as  advocated  by  Bernhardt  (1991) , 
consists  of  the  students  reviewing  the  texts  until  they  believe  they 
have  gathered  as  much  information  from  the  texts  as  they  can.  There 
is  no  time  limit  on  this  phase.  Once  they  are  finished  reading  the 
texts  they  write  down  as  much  about  the  passages  as  they  can 
remember  without  referring  back  to  the  text.  The  students  are  to 
write  in  their  LI  in  order  to  maximize  their  ability  to  describe  * 
what  they  understand.  The  protocols  are  then  scored.  Bernhardt  has 
found  the  procedure  provides  a  more  detailed  and  accurate  picture  of 
what  has  been  comprehended  by  the  student.  Johnston  (1983)  provides 
additional  endorsement  citing  the  straightforwardness  and  richness 
of  the  data  that  resulted  from  the  protocols. 

The  two  most  common  scoring  procedures  are  those  of  Meyer 
(1985)  and  Johnson  (1970)  .  Both  are  effective  is  assessing 
comprehension,  but  Bernhardt  (1991)  determined  that  the  use  of 
Johnson's  procedure  was  more  efficient  overall.  Johnson's  procedure 
is  based  upon  dividing  up  the  text  into  pausal  units  and  assigning 
values  to  these  units  based  upon  their  relative  importance  to  the 
text.  The  summing  of  the  propositions  understood  by  the  subject 
becomes  the  comprehension  score. 

Selection  of  Texts 

The  three  texts  selected  were  ones  that  had  been  used 
successfully  in  studies  in  the  past,  specifically  the  study 
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conducted  by  Heinz  at  the  Air  Force  Academy  in  1993  examining 
automated  recall  scoring.  The  texts  are  in  German  and  the  subjects 
were  cadets  enrolled  in  German  classes.  Two  are  brief  reports  on 
items  of  interest  and  the  other  is  a  letter — all  are  authentic  in 
the  sense  that  they  were  written  in  unsimplified  German  for  actual 
use  and  remained  unaltered.  The  texts  are  relatively  brief,  between 
150  and  200  words,  and  are  not  overly  complex  in  terms  of  writing  or 
content.  A  preliminary  study  was  conducted  in  May  1996  to  determine 
the  suitability  of  these  texts  for  use  in  the  current  study.  The 
results  indicated  that  both  beginning  and  intermediate  learners 
could  comprehend  a  reasonable  portion  of  the  text,  though  there  was 
considerable  variation.  This  result  is  in  agreement  with  Heinz' 
results,  which  indicated  that  all  students  could  access  some  of  the 
information  and  that  the  higher  scores  corresponded  with  the  higher 
levels  of  German  classes.  The  preliminary  study  also  found  that 
both  beginning  and  intermediate  German  students  felt  that  they  knew 
many  of  the  words  in  the  text.  Given  the  results  of  this 
preliminary  study,  as  well  as  the  data  gathered  by  Heinz,  it  was 
decided  that  the  texts  were  appropriate  and  useable  for  this 
experiment . 


Selection  of  Words 

The  bank  of  words  for  the  study  consists  of  the  words  not  known 
by  any  of  the  students  in  the  preliminary  study.  Preference  was 
given  to  words  that  are  widely  applicable  or  common.  In  addition, 
the  following  criteria  were  applied: 

.  -  Six  words  were  used  from  each  of  the  texts 
-  The  words  were  not  instructed  in  class  either  prior  to  or 
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during  the  study 

-  The  words  were  confirmed  as  unknown  on  a  pretest 
Several  additional  words  were  chosen  from  each  text,  in  the  event 
some  of  the  cadets  have  knowledge  of  the  targeted  words.  It  proved 
unnecessary  to  use  the  alternate  words. 

Procedures  and  Data  Collection 

Pre-experiment 

Two  weeks  prior  to  the  experiment  the  subjects  were  given  a 
checklist  of  words  that  included  the  targeted  vocabulary  as  well  as 
some  distracter  words  they  should  have  known.  The  cadets  were  asked 
to  indicate  whether  or  not  they  knew  a  word.  Based  upon  the  outcome 
of  this  test,  six  words  were  chosen  from  each  text  to  form  the  final 
eighteen  targeted  words. 

Experiment 

The  actual  experiment  consisted  of  a  four-day  cycle  and  took 
place  in  March  1997.  The  cycle  was  familiarization,  instruction, 
followed  by  recall  protocols,  and  ending  with  vocabulary  testing. 

On  the  first  day,  the  students  went  to  the  Language  Learning 
Center  and  went  through  two  recalls  utilizing  the  computerized 
recall  procedure.  The  intent  of  this  was  to  familiarize  the 
subjects  with  the  computer  program  and  the  overall  process.  None  of 
the  data  collected  during  this  period  was  utilized. 

On  the  second  day,  the  students  proceeded  to  the  Language 
Learning  Center  during  their  normally  scheduled  class.  The  selected 
students  (half  of  the  total  subjects)  received  computerized 
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instruction  on  the  18  targeted  vocabulary  words.  The  other  half 
were  instructed  on  non-targeted  words  in  the  text.  The  instruction 
was  be  self-paced  and  allowed  for  ample  review.  The  only 
restriction  was  the  overall  time  limit  of  45  minutes. 

On  day  three,  all  the  cadets  went  to  the  Language  Learning 
Center  and  took  their  places  at  the  assigned  computers.  They  then 
completed  recall  protocols  on  the  three  selected  texts.  Each  text 
was  shown  on  the  screen  until  the  student  clicked  that  he  or  she  was 
ready  to  write  the  protocol.  At  that  point  the  text  disappeared  and 
the  cadets  were  to  type  everything  that  was  understood  and 
remembered.  Once  the  protocol  was  completed  the  next  text  appeared 
on  the  screen.  The  pattern  was  repeated  until  all  texts  are 
completed.  During  this  phase,  half  of  the  subjects  had  access  to 
on-line  dictionaries  and  could  access  the  definition  of  any  word  by 
highlighting  it  on  the  computer  screen.  There  was  no  time  limit  for 
each  protocol,  but  rather  an  overall  time  limit  of  45  minutes  to 
finish  all  three  protocols. 

On  the  final  day,  all  students  once  again  reported  to  the 
Language  Learning  Center  for  a  vocabulary  test.  The  test  was  given 
on  the  computer  and  consisted  of  word  identification  task  and  a 
supply-definition  task  for  all  of  the  targeted  vocabulary.  Once  the 
test  was  completed,  the  students  were  asked  to  fill  out  a  brief 
survey  on  the  computer  concerning  their  feelings  about  computerized 
instruction  and  testing. 

Delayed  Testing 

Two  weeks  after  the  testing  of  the  vocabulary  knowledge,  the 
students  returned  and  repeated  the  test  in  order  to  measure 
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vocabulary  retention  and  the  effect  on  both  automaticity  and  word 
knowledge. 


Pilot  Study 

A  pilot  study  was  conducted  in  February  of  1997  to  allow  for 
fine-tuning  of  the  procedures  to  be  used  for  the  study.  The  pilot 
study  involved  classes  of  German  cadets  similar  to  those  who  took 
part  in  the  actual  experiment.  The  purpose  of  the  pilot  study  was 
to  check  the  functioning  of  the  computer  programs,  data  collection, 
and  the  viability  of  the  overall  process.  The  pilot  study  covered 
two  days  in  which  the  students  went  through  the  entire  process 
looking  for  flaws.  With  the  exception  of  several  minor  timing  and 
on-screen  presentation  problems  that  were  noted,  the  overall  process 
worked  well.  The  identified  problems  were  alleviated  in  subsequent 
programming . 

Null  Hypotheses 

The  null  hypotheses  are  organized  under  their  appropriate  research 
questions. 

HOI:  There  will  be  no  overall  significance  for  the  MANCOVA. 

H02 :  There  v/ill  be  no  significant  effect  for  the  covariant  of 

verbal  skill . 

H03 :  There  will  be  no  significant  main  effect  for  focus  and  non¬ 

focus  groups. 
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Subset  1  Is  there  a  significant  difference  between  the  focus  and 


the  non-focus  group  as  measured  by  level  of  knowledge,  automaticity, 
and  reading  comprehension? 

H03a  There  will  be  no  significant  difference  between  the 

focus  and  the  non- focus  group  on  the  level  of  knowledge 
results . 

H03b  There  will  be  no  significant  difference  between  the 
focus  and  the  non- focus  groups  on  the  automaticity 
results. 

H03c  There  will  be  no  significant  difference  between  the 

focus  and  non-focus  group  on  the  recall  protocol  scores. 

H04:  There  will  be  no  significant  main  effect  for  the  lexical 

access  vs.  non-lexical  access  groups. 

Subset  2  Is  there  a  significant  difference  between  the  groups  with 
lexical  access  and  the  groups  without  lexical  access  as  measured  by 
level  of  knowledge,  automaticity,  and  reading  comprehension? 

H04a  There  will  be  no  significant  difference  between  the 

groups  with  and  without  lexical  access  on  the  level  of 
knowledge  results. 
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H04b 


There  will  be  no  significant  difference  between  the 
groups  with  and  without  lexical  access  on  the 
automaticity  results. 

H04c  There  will  be  no  significant  difference  between  the 
groups  with  and  without  lexical  access  on  the  recall 
protocol  scores . 

H05:  There  will  be  no  significant  main  effect  for  interaction. 

Subset  3  Is  there  significant  interaction  between  group  (focus /non¬ 
focus)  and  lexical  access  (yes/no)  on  level  of  knowledge, 
automaticity,  and  reading  comprehension? 

H05a  There  will  be  no  significant  interaction  between  focus 
and  lexical  access  on  level  of  knowledge  results. 

H05b  There  will  be  no  significant  interaction  between  focus 
and  lexical  access  on  automaticity  results. 

H05c  There  will  be  no  significant  interaction  between  focus 
and  lexical  access  on  the  recall  protocol  scores. 
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Is  there  a  significant  difference  between  immediate  and  delayed 
scores  on  depth  of  knowledge  and  automaticity? 

H06  There  will  be  no  significant  difference  between 
immediate  and  delayed  test  scores  for  depth  of 
knowledge. 

H07  There  will  be  no  significant  difference  between 

Immediate  and  delayed  test  scores  for  automaticity. 
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CHAPTER  4 


RESULTS  AND  DISCUSSION 

Introduction 

The  recent  trend  of  avoiding  direct  instruction  of  definitions 
has  been  brought  into  question  by  a  number  of  recent  studies.  This 
study  investigates  whether  there  are  advantages  in  deliberate 
advance  instruction  of  definitions  on  the  learning  of  word  meanings 
and  overall  understanding  of  texts  containing  the  words. 

A  group  of  130  cadets  were  randomly  placed  into  one  of  four 
groups  varying  on  the  two  independent  variables  of  lexical  access 
and  prior  instruction  of  the  targeted  words.  The  subjects  took  part 
in  a  four-day  process  of  familiarization  with  the  computer  programs, 
vocabulary  instruction,  reading  comprehension,  and  vocabulary 
testing.  A  delayed  test  was  administered  two  weeks  later.  One 
hundred  cadets  were  randomly  selected  from  this  group  for  the  final 
analysis.  A  Multivariate  Factorial  Analysis  of  Covariance  was 
conducted  with  three  dependent  variables  —  level  of  knowledge, 
automaticity  of  access,  and  reading  comprehension.  Delayed  scores 
were  also  compared  with  the  primary  scores  looking  for  differences 
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and  patterns.  For  roughly  one  third  of  the  participants  scores  were 
available  for  the  two  sub-skill  areas  of  word  knowledge  and  reading 
comprehension.  Further  analyses  were  conducted  using  these  two  sub¬ 
skill  areas  as  covariant  against  the  appropriate  dependent 
variables. 

Affective  variables  were  examined  in  two  ways.  An  on-line 
survey  was  given  upon  the  completion  of  the  research  concerning 
their  impression  of  the  study  and  the  effectiveness  of  the  process. 
The  second  method  was  the  selection  of  one  subject  per  group  for  a 
more  in-depth  study.  The  study  of  these  four  cadets  provided 
insights  that  might  have  otherwise  escaped  attention. 

Data  Analysis 

The  purpose  of  this  chapter  is  to  report  on  the  preparation 
and  analyses  of  the  data.  The  quantitative  aspects  will  be  dealt 
with  first,  followed  by  the  qualitative  results.  The  quantitative 
portion  will  be  divided  up  into  the  following  sections: 

*  Data  Preparation 

*  Descriptive  Data 

*  MANCOVA 

-  Overall  significance  and  effect  of  the  covariant 

-  Main  Effects 

-  Differences  between  focus  and  non^focus  groups 

-  Differences  between  those  with  and  without  lexical 
access 

-  Interaction  between  group  and  lexical  access 

*  Differences  between  immediate  and  delayed  scores 
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The  qualitative  portion  will  be  split  in  the  following  manner: 

*  Survey  questions 

*  Case  studies 


Quantitative  Data 


Data  Preparation 

In  turning  the  raw  data  into  data  that  could  be  conveniently 
analyzed,  it  became  apparent  that  some  additional  programming  could 
have  simplified  this  matter  greatly.  An  appropriate  spreadsheet 
could  have  accomplished  in  minutes  what  it  took  weeks  to  complete  by 
hand.  The  information  gathered  on  level  of  knowledge,  recalls,  and 
automaticity  all  required  additional  manipulation  to  ready  them  as 
data  to  be  analyzed. 

The  level  of  knowledge  scores  for  individual  items  were 
examined  by  the  researcher,  and  where  conflicts  existed  between 
raters  they  were  worked  out  between  the  appropriate  raters.  Of 
note,  there  were  relatively  few  discrepancies  between  the  raters  — 
on  many  pages  none,  and  rarely  more  than  two  per  subject.  Once  the 
individual  scores  had  been  checked,  they  were  tallied  for  each 
individual  and  the  totals  were  then  used  in  the  overall  analysis. 

The  recall  protocols  were  scored  via  Dr.  Heinz's  automatic 
scoring  procedures.  The  resulting  spreadsheet  data  was  examined  for 
any  procedural  or  logical  errors,  as  well  as  spot  checking  for 
general  agreement  with  manual  scoring.  The  process  worked  as 
smoothly  as  it  had  for  Dr.  Heinz  during  his  own  research.  The  raw 
scores  were  summed  and  the  totals  submitted  for  the  analysis. 

Calculating  an  average  time  score  for  the  automaticity  factor 
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proved  more  problematic.  Although  averaging  times  is  a 
straightforward  procedure,  the  problem  was  that  many  of  the  cadets 
had  instances  of  time  scores  that  were  statistical  outliers.  This 
happened  either  through  distraction  (talking  with  a  neighboring 
cadet)  or  bouts  of  distraction.  A  random  sub-sample  of  twelve 
subjects  was  taken  and  analyzed.  The  result  average  was  7.82 
seconds  with  a  standard  deviation  of  3.62  seconds.  This  meant  that 
any  scores  above  18.68  seconds  were  unlikely  to  come  from  the  real 
population.  This  time  frame  also  passes  the  common  sense  test,  it 
being  unrealistic  to  believe  that  more  than  twenty  seconds  is 
necessary  to  ponder  the  extent  of  one's  word  knowledge.  A  cutoff 
time  of  20  seconds  was  used,  with  any  scores  greater  than  this  time 
reduced  to  twenty  seconds.  Once  these  corrections  had  been  made, 
the  times  for  each  subject  were  averaged  and  these  averages  used  for 
further  analysis. 

Descriptive  Data 

The  section  examines  the  descriptive  data  resulting  from  the 
study.  The  first  data  set  will  be  that  concerning  the  dependent 
variable  of  word  knowledge,  followed  by  the  results  for 
automaticity,  and  then  the  recall  protocols.  Lastly,  the  delayed 
test  scores  for  both  level  or  word  knowledge  and  automaticity  are 
presented.  The  scores  of  the  dependent  variables  for  the 
experiments  are  arranged  by  means  and  standard  deviations. 
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Table  2  presents  the  level  of  knowledge  scores  from  the 
immediate  test.  Of  note  is  the  fact  that  the  overall  mean  for  the 
group  that  had  been  instructed  in  the  target  words  (M  =  32.22)  is 
much  higher  than  those  subjects  instructed  in  the  non-targeted  words 
(M  =  9.38).  Although  the  mean  for  those  subjects  with  lexical 
access  (M  =  21.96)  is  higher  than  for  those  subjects  lacking  such 
access  (M  =  19.64),  the  difference  is  considerably  smaller  than  that 
evidenced  above.  The  difference  between  the  non-focus  groups, 
however,  is  quite  striking,  with  the  group  having  lexical  access 
(M  =  11.52)  roughly  40%  higher  than  the  group  without  access 
(M  =  7.24) . 


Non-Focus 

Focus 

Overall 

M 

SD 

M 

SD 

M 

SD 

No  Lexical  Access 

7.24 

4.70 

32.04 

9.11 

19.64 

14.44 

Lexical  Access 

11.52 

5.90 

32.40 

8.03 

21.96 

12.68 

Overall 

9.38 

5.71 

32.22 

8.55 

20.80 

13.57 

Table  2 

Means  and  Standard  Deviations 

of  Level 

of  -  Vocabulary  Knowledge 

Scores  as  a  Function 

of  Group 

( focus /non-focus )  and 

Lexical  Access 
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Table  3  presents  the  descriptive  data  for  the  two  independent 
variables  on  the  time  of  response.  The  overall  focus  mean  (M  = 
6.00)  was  faster  than  the  overall  non-focus  mean  (M  =  6.53). 

Whereas  the  faster  response  of  the  focus  group  was  expected,  the 
following  information  was  counter  to  expectations.  The  mean  of  the 
response  times  for  those  with  lexical  access  was  (M  =  6.75)  was 
slower  than  the  mean  for  those  without  access  (M  =  5.77). 


Non-Focus  Focus  Overall 


M 

SD 

M 

SD 

M 

SD 

No  Lexical  Access 

5.76 

2.46 

5.79 

1.85 

5.77 

2.15 

Lexical  Access 

7.30 

2.14 

6.21 

1.84 

6.75 

2.05 

Overall 

6.53 

2.41 

6.00 

1.84 

6.26 

2.15 

Table  3 

Means  and  Standard  Deviations  of  Response  Times  as  a  Function  of 
Group  ( focus/non-focus )  and  Lexical  Access 
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Table  4  reports  the  descriptive  data  for  the  recall  protocols 
on  the  immediate  test.  The  subjects  with  lexical  access  (M  =  96.80) 
outscored  those  with  no  lexical  access  (M  =  86.66).  Interestingly, 
the  non-focus  group  scored  higher  in  all  situations.  This 
distinction  can  be  clearly  seen  in  Figure  1,  which  clearly  indicates 
that  Group  2,  the  non- focus  group  with  lexical  access,  is  superior 
to  any  of  the  others. 


Non-Focus  Focus  Overall 


M 

SD 

M 

SD 

M 

SD 

No  Lexical  Access 

89.12 

39.08 

84.20 

27.23 

86.66 

33.43 

Lexical  Access 

102.00 

36.99 

91.60 

27.55 

96.80 

32.70 

Overall 

95.56 

38.22 

87.90 

27.37 

91.73 

33.29 

Table  4 

Means  and  Standard  Deviations  of  Recall  Protocol  Scores  as  a 
Function  of  Group  (focus/non-focus)  and  Lexical  Access 
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Delayed  Scores 

The  delayed  scores  were  gathered  two  weeks  after  the  initial 
testing  period  via  identical  procedures.  The  same  process  was  used 
in  gathering,  grading,  and  preparing  the  data.  This  phase  of  the 
research  concerned  itself  with  the  word  knowledge,  so  recall 
protocols  were  not  part  of  the  evaluation. 

Table  5  presents  the  level  of  knowledge  scores  on  the  delayed 
test.  In  general,  the  means  reflect  the  same  general  pattern  as  was 
seen  on  the  immediate  test,  with  the  means  for  the  focus  group 
(M  =  24.68)  being  considerably  higher  than  those  of  the  non-focus 
group  (M  =  9.00). 


Non-Focus  Focus  Overall 


M 

SD 

M 

SD 

M 

SD 

No  Lexical  Access 

6.32 

4.62 

23.86 

8.76 

15.09 

11.26 

Lexical  Access 

11.68 

6.40 

25.50 

7.91 

18.59 

9.97 

Overall 

9.00 

6.15 

24.68 

8.29 

16.84 

10.72 

Table  5 

Means  and  Standard  Deviations  of  Level  of  Knowledge  Vocabulary 
Scores  as  a  Function  of  Group  ( focus/non-focus )  and  Lexical  Access 
on  the  Delayed  Test 
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Table  6  depicts  the  time  responses  from  the  delayed  test.  As 
was  the  case  with  the  knowledge  scores,  the  pattern  of  the  responses 
corresponds  closely  with  the  pattern  on  the  immediate  test.  In 
fact,  the  values  are  virtually  the  same  as  they  were  on  the 
immediate  test. 


Non-Focus  Focus  Overall 


M 

SD 

M  SD 

M 

SD 

No  Lexical  Access 

5.87 

2.51 

5.35  1.75 

5.61 

2.15 

Lexical  Access 

7.18 

2.79 

6.06  2.02 

6.62 

2.48 

Overall 

6.53 

2.71 

5.70  1.90 

6.12 

2.36 

Table  6 

Means  and  Standard  Deviations 

of  Response  Times  as 

a  Function  of 

Group  (focus/non-focus) 

and  Lexical 

Access  on  the 

Delayed 

Test 
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The  differences  between  immediate  and  delayed  scores  can  best 
be  illustrated  graphically,  as  seen  in  Figure  2.  The  superiority  of 
Group  1  and  3,  both  of  which  had  the  prior  definitional  instruction, 
in  both  immediate  and  delayed  testing  is  readily  apparent.  Although 
there  is  a  decline  in  level  of  knowledge  over  the  course  of  two 
weeks,  even  the  delayed  scores  of  these  two  groups  tower  over  the 
level  of  knowledge  scores  of  Groups  0  and  2. 

Also  of  interest  is  the  apparent  difference  between  groups 
zero  and  two.  These  are  the  two  groups  not  instructed  in  the 
targeted  vocabulary,  where  group  two  had  lexical  access  while 
reading  the  passages  and  group  zero  had  no  additional  assistance. 

The  scores  for  the  group  with  lexical  access  are  almost  double  those 
for  the  group  without  access,  suggesting  a  possible  interaction 
between  instruction  and  lexical  access. 
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Note:  f =focus ;  nf =non-focus ;  l=lexical  access;  nl=no  lexical  access 


Figure  1 

Means  of  the  Immediate  and  Delayed  Level  of  Knowledge  Scores  by 
Group 
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In  a  similar  graphical  comparison,  Figure  3  displays  both 
immediate  and  delayed  automaticity  data.  What  is  striking  about 
this  chart  is  the  virtual  lack  of  variation  amongst  the  groups. 
Also  remarkable  is  that  there  is  a  decrease  in  reaction  time 
(theoretically  indicating  an  increase  in  automaticity)  from  the 
initial  test  in  three  of  the  four  groups. 


Figure  2 

Means  of  the  Immediate  and  Delayed  Automaticity  Scores  by  Group 
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Inferential  Data 


A  MANCOVA  analysis  was  conducted  in  order  to  answer  the  basic 
research  hypotheses  associated  with  the  study.  Table  7  will  be  used 
to  examine  null  hypotheses  one  through  five,  and  Table  8  will  be 
used  on  for  the  more  detailed  examination  of  the  subset  null 
hypotheses.  Hypotheses  6  and  7  were  examined  using  simple  t-tests. 


Effect 

Value 

df 

F 

p 

Power 

Overall 

.865 

3 

4.828 

.004 

.893 

Verbal  Skill 

.995 

3 

.152 

.928 

.077 

Group 

.240 

3 

93.187 

.000 

1.000 

Lexical  Access 

.921 

3 

2.654 

.053 

.631 

Group  *  Lexical 

.966 

3 

1.088 

.358 

.286 

Table  7 

Summary  Table  for 

the  Main 

Effects 

o  f  the  MANCOVA 
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Hoi:  There  will  be  no  overall  significance  for  the  MANCOVA. 

This  hypothesis  must  be  rejected  on  the  basis  of  the  data. 

The  F(3,  93)  =  4.828,  £  =  .004  indicates  an  overall  significance 
within  the  MANCOVA. 

Ho2 :  There  will  be  no  significant  effect  for  the  covariant  of 

verbal  skill. 

This  hypothesis  cannot  be  rejected.  Verbal  skill  was  shown  to 
have  negligible  impact  with  an  F(3,  93)  =  .152  and  £  =  .928. 

Ho3:  There  will  be  no  significant  main  effect  for  group. 

This  hypothesis  must  be  rejected  on  the  basis  of  the  data. 

The  MANCOVA  resulted  in  an  F(3,  93)  =  98.187,  with  a  £  =  .000.  The 
observed  power  associated  with  this  result  was  1.000. 

Ho4 :  There  will  be  no  significant  main  effect  for  lexical 

access . 

This  hypothesis  cannot  be  rejected.  The  data  indicated  that 
the  main  effect  for  lexical  access  was  just  shy  of  significance  with 
an  F ( 3,  93)  =  2.654,  £  =  .053.  Given  the  level  of  significance  and 
the  mediocre  observed  power,  the  area  of  lexical  access  is  worthy  of 
closer  investigation. 

Ho5:  There  will  be  no  significant  interaction  between  the  two 

main  effects. 

This  hypothesis  cannot  be  rejected  on  the  basis  of  the  data. 
The  MANCOVA  revealed  an  F(3,  93)  =  1.088  and  a  £  =  .358.  The 
observed  power  for  this  interaction  is  very  low  at  .286. 
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The  table  below  contains  the  results  of  the  univariate 
analyses  and  will  be  used  to  address  the  subset  hypotheses 
concerning  individual  dependent  variables. 


Source 

Dep  Var 

df 

MS 

F 

P 

Knowledge 

1 

10.536 

.203 

.653 

Verbal 

Response  Time 

i 

2.587E- 

■02  .006 

.939 

Skills 

Recalls 

1 

120.255 

.108 

.743 

Knowledge 

i 

12972.583 

249.714 

.000 

Group 

Response  Time 

1 

6.997 

1.589 

.210 

Recalls 

1 

1540.505 

1.389 

.242 

Knowledge 

1 

137.879 

2.654 

.107 

Lexical 

Response  Time 

1 

24.222 

5.503 

.021 

Access 

Recalls 

1 

2511.658 

2.264 

.136 

Group  * 

Knowledge 

1 

98.056 

1.888 

.173 

Lexical 

Response  Time 

1 

7.852 

1.784 

.185 

Access 

Recalls 

1 

177.650 

.160 

.690 

Knowledge 

95 

51.950 

Error 

Response  Time 

95 

4.402 

Recalls 

95 

1109.372 

Knowledge 

99 

Total 

Response  Time 

99 

Recalls 

99 

Table  8 

Summary  Table  for  the  Univariate  Analyses 
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Ho3a:  There  will  be  no  significant  difference  between  the 

focus  and  non-focus  groups  on  the  level  of  knowledge  scores . 

This  hypothesis  must  be  rejected  on  the  basis  of  the  data. 

The  non-focus  mean  of  9.38  is  far  lower  than  the  focus  mean  of  32.22 
(see  Table  1) .  The  results  of  the  MANCOVA  revealed  that  the 
difference  was  significant  F(l,  95)  =  249.71,  £  <  .000. 

Ho3b :  There  will  be  no  significant  difference  between  the 

focus  and  non-focus  groups  on  the  automaticity  scores. 

This  hypothesis  cannot  be  rejected.  The  difference  between 
the  focus  group  mean  of  6.00  and  the  non-focus  mean  of  6.53,  though 
indicating  a  slightly  faster  response  time,  failed  to  be 
statistically  significant.  The  MANCOVA  resulted  in  an  F(l,  95)  = 
1.59,  £  =  .210. 

Ho3c:  There  will  be  no  significant  difference  between  the 

focus  and  non-focus  groups  on  the  recall  protocol  scores. 

This  hypothesis  cannot  be  rejected.  The  difference  between 
the  focus  group  mean  of  51.34  and  the  non-focus  group  mean  of  53.23 
was  found  to  be  non-significant.  The  MANCOVA  results  indicated  an 
F ( 1,  95)  =  1.39,  £  =  .242. 

Ho4a :  There  will  be  no  significant  difference  between  those 

with  and  without  lexical  access  on  the  level  of  knowledge  scores. 

This  hypothesis  cannot  be  rejected.  The  group  mean  for  those 
with  lexical  access  was  21.96  compared  with  19.64  for  those  without 
lexical  access.  The  MANCOVA  produced  an  F(l,  95)  =  2.65,  £  =  .107; 
not  significant  at  the  .05  level  of  significance,  but  still  a 
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difference  worthy  of  some  degree  of  attention.  Upon  closer 
inspection,  however,  it  is  apparent  that  if  one  looks  specifically 
at  the  group  who  received  no  instruction  in  the  target  words  there 
appears  to  be  a  considerable  effect.  Those  subjects  without  lexical 
access  had  a  mean  score  of  7.24,  whereas  those  with  lexical  access 
achieved  a  mean  of  11.52.  This  will  be  discussed  further  when 
dealing  with  Ho5a. 

Ho4b :  There  will  be  no  significant  difference  between  those 

with  and  without  lexical  access  on  the  automaticity  scores. 

This  hypothesis  can  be  rejected  on  the  basis  of  the  data. 

Those  without  lexical  access  averaged  a  response  time  of  5.77 
seconds,  considerably  faster  the  group  mean  of  6.75  seconds  for 
those  with  the  lexical  access.  The  MANCOVA  displayed  an  F(l,  95)  = 
5.50, £=.021. 


K°4c :  There  will  be  no  significant  difference  between  those 

with  and  without  lexical  access  on  the  recall  protocol  scores. 

This  hypothesis  cannot  be  rejected.  The  group  mean  for  those 
with  lexical  access  was  53.58  as  compared  with  those  without  lexical 
access  of  50.99.  The  MANCOVA  showed  an  F(l,  95)  =  2.26,  £  =  .136; 
once  again  not  significant  at  the  .05  level,  but  yet  not  to  be 
completely  disregarded. 
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Ho5a:  There  will  be  no  significant  interaction  between  group 


(focus  vs  non-focus)  and  lexical  access  on  the  level  of  knowledge 
scores . 

This  hypothesis  could  not  be  rejected  on  the  basis  of  the 
initial  data.  The  MANCOVA  revealed  no  significant  interaction 
between  lexical  access  and  group  (focus/non-focus)  with  an  F(l,  95) 

=  1.88,  p  =  .173.  Upon  closer  inspection  of  the  data  and  Figure  2, 

*■ 

however,  it  is  clearly  the  case  that,  in  the  absence  of  instruction, 
the  group  with  lexical  access  was  well  superior  to  the  group  with  no 
lexical  access. 

This  clear  discrepancy  between  these  two  groups  justified  a 
further  analysis  of  the  data.  Consequently,  a  simple  two  sample 
Student's  t-test  was  run  on  these  two  non-focus  groups  resulting  in 
a  high  degree  of  significance  (p  =  .004).  The  importance  of  lexical 
access  did  not  become  apparent  during  the  MANCOVA  due  to  the 
overpowering  impact  of  instruction,  which  served  to  mitigate  the 
effects  described  just  above.  This  dramatic  difference  suggests 
that,  in  the  absence  of  any  vocabulary  instruction,  lexical  access 
is  clearly  helpful  in  furthering  word  knowledge .  Examining  Figure  3 
provides  additional  support  for  this  claim  because  the  lines,  though 
not  having  a  dramatic  intersection,  do  intersect.  The  point  should 
be  made  that,  though  the  relationship  appears  linear  in  Figure  3, 
the  relationship  is  likely  far  more  complex  and  may  vary 
considerably  depending  upon  the  type  and  degree  of  instruction  that 
is  being  provided. 
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Figure  3 

Graphical  representation  of  the  interaction  between  lexical  access 

{ 

and  group 
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Ho5b :  There  will  be  no  significant  interaction  between  group 

and  lexical  access  on  automaticity  results. 

This  hypothesis  cannot  be  rejected.  The  MANCOVA  for  the 
interaction  resulted  in  a  value  of  F(l,  95.)  =  1.78,  £  -  .185; 
considerably  less  than  the  critical  value. 


—4 — No  Lexical  Access 
—0 — Lexical  Access 


Figure  4 

Graphical  representation  of  the  interaction  between  lexical  access 
and  group 
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Ho5c:  There  will  be  no  significant  interaction  between  group 

and  lexical  access  on  the  recall  protocol  scores. 

This  hypothesis  also  cannot  be  rejected.  The  interaction 
between  lexical  access  and  group  was  virtually  non-existent 
resulting  in  an  F(l,  95)  =  .160,  p  =  .690. 


•No  Lexical  Access 
-  Lexical  Access 


Figure  5 

Graphical  representation  of  the  interaction  between  lexical  access 
and  group 
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Ho6 :  There  will  be  no  significant  difference  between 


immediate  and  delayed  test  scores  for  level  of  knowledge. 

This  hypothesis  can  be  rejected.  A  Student's  two  sample  t- 
test  was  performed  with  a  resulting  £  =  .026.  This  significant 
difference  is  the  expected  result  of  a  two-week  delay  prior  to 
administering  the  second  test. 

Ho7 :  There  will  be  no  significant  difference  between 

immediate  and  delayed  test  scores  for  automaticity . 

This  hypothesis  cannot  be  rejected.  The  analysis,  utilizing 
Student's  t-test  resulted  in  a  £  =  .660  indicating  a  fairly  close 
match  between  the  two  sets  of  scores. 


Affective  Data 

The  subject  responses  below  we  re  gathered  on  the  computer  once 
they  immediate  word  knowledge  test  had  been  completed.  There  was  no 
time  limit  and  the  students  were  encouraged  to  be  as  forthcoming  as 
possible . 

Question  #1 

What  was  your  general  impression  of  the  study? 

Although  the  responses  varied,  there  were  several  consistent 
strands  that  ran  through  all  of  them.  Many  students  voiced  the 
opinion  that  the  study  was  well  organized  and  that  the  overall 
process  flowed  smoothly.  The  two  descriptors  that  were  advanced 
most  frequently  were  good  and  interesting .  The  consensus  was  that 
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working  with  the  computers  was  an  enjoyable  and  worthwhile 
environment.  Another  consensus  was  that  the  overall  level  of  the 
materials  was  tough,  perhaps  too  advanced.  This  level  of  difficulty 
frustrated  some  of  the  subjects,  but  did  not  seem  to  bother  others 
who  viewed  it  as  a  challenge.  In  an  interesting  contradiction,  some 
cadets  mentioned  that  the  study  showed  how  much  German  they  actually 
knew,  while  others  noted  that  it  laid  bare  their  lack  of  knowledge. 

Question  #2 

Do  you  feel  this  type  of  vocabulary  instruction  might  be  beneficial? 

The  answers  to  this  question  were  virtually  all  in  the 
affirmative,  though  they  varied  in  the  degree  of  enthusiasm.  There 
were  some  negative  responses,  but  they  were  relatively  few.  The 
responses  ranged  from  "absolutely  the  best  vocabulary  instruction  I 
have- seen"  to  "the  older  way  is  better".  A  widely  voiced  opinion 
was  that  it  would  have  been  better  with  more  cycles  of  repetition, 
and  generally  more  exposure  to  the  words .  In  terms  of  an  initial 
exposure,  most  of  the  cadets  felt  they  learned  well  utilizing  this 
type  of  program.  The  overall  response  to  this  modified  flash  card 
approach  was  overwhelmingly  positive. 

Question  #3 

What  was  your  impression/assessment  of  the  computer  reading 
comprehension  program? 

The  impressions  about  the  reading  comprehension  were  also 
positive  in  general  but  also  had  many  more  negative  comments.  Many 
of  these  negative  comments,  however,  were  in  reference  to  the  level 
of  difficulty  of  the  articles  rather  than  inherent  problems  with  the 
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program  or  the  process.  Many  felt  that  the  articles  were  too  hard 
and  they  found  that  frustrating.  The  common  feeling,  even  among 
those  who  believed  they  did  poorly,  was  that  the  program  would 
provide  an  accurate  reflection  of  their  level  of  knowledge.  There 
were  also  some  negative  comments  about  the  dictionary  function 
stating  that  it  would  be  more  useful  if  it  were  easier  to  use, 
specifically  one  should  simply  have  to  click  on  a  word  rather  than 
having  to  scan  through  a  list. 

At  this  point,  a  bit  of  additional  elaboration  concerning  the 
dictionary  function  is  necessary.  In  their  normal  computer 
environment  in  the  language  lab,  they  have  merely  to  click  on  a  word 
and  the  English  meaning  is  displayed.  The  procedure  of  having  to 
scroll  through  a  list  naturally  seems,  by  comparison,  quite  tedious 
and  troublesome.  Some  of  the  subjects  did  not  make  much  use  of  the 
dictionary  function,  and  this  may  serve  as  a  possible  explanation. 
The  list  approach  was  utilized  because  the  necessary  programming  was 
more  achievable,  and  also  because  it  more  closely  resembled  process 
of  using  a  normal  dictionary.  This  aspect  of  the  programming  will 
be  discussed  again  at  a  later  point. 

Question  #4 

What  was  your  impression/assessment  of  the  computer  vocabulary 
program? 

Although  this  question  was  targeted  more  at  the  technical 
aspects  of  the  program  than  the  instruction  (question  #2)  the 
responses  for  this  question  generally  echoed  those  in  question  #2. 
Once  again,  the  responses  were  virtually  all  positive,  but  a 
distinction  was  made  in  this  section  that  was  not  present 
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previously,  that  between  the  vocabulary  instruction  program  and  the 
vocabulary  assessment  program.  The  negative  opinions  were  generally 
aimed  at  the  evaluation  portion  of  the  study,  though  few  specifics 
were  given. 

The  calls  for  more  repetitions  of  the  process,  resulting  in 
greater  exposure  to  the  words,  were  also  evidenced  in  this  question. 
These  requests  for  continued  training  were  usually  accompanied  by 
observations  concerning  retention,  and  so  may  not  have  been  directly 
related  to  the  program  itself.  Another  point  of  note  in  virtually 
all  of  the  responses  to  this  question,  as  well  as  the  other 
questions,  is  that  the  cadets  seem,  with  few  exceptions,  to  be 
completely  comfortable  using  and  interacting  with  the  computers.  In 
one  case,  a  student  wrote  that  he  "felt  more  comfortable  with  a 
machine  than  an  instructor". 


Question  #5 

Did  you  feel  that  the  words  you  learned  were  helpful  to  you  in 
comprehending  the  reading  passages? 

The  cadets  were  split  on  this  question  with  a  slight  majority 
believing  that  the  learned  words  were  at  least  somewhat  helpful. 

Even  those  who  felt  the  words  were  helpful,  however,  usually 
qualified  their  statements  to  the  effect  that,  though  the  words 
helped  establish  the  gist  of  the  passages,  the  learned  words  were 
too  few  to  be  of  assistance  in  deriving  the  details.  On  the  other 
extreme,  there  were  several  subjects  who  felt  that  the  words  were  of 
no  help  at  all,  citing  that  too  much  of  the  remaining  text  was 
unknown . 
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These  responses  amplify  the  complaints  voiced  in  that  the 
texts  were  of  relatively  high  difficulty  given  the  level  of  the 
learner.  Although  these  types  of  comments  were  not  completely 
unexpected  given  the  nature  of  the  research,  the  small  degree  of 
assistance  rendered  by  the  targeted  words  was  surprising, 
particularly  in  the  groups  that  had  dictionary  access  to  fill  in  the 
missing  blanks. 

Question  #6 

How  well  were  you  able  to  retain  the  instructed  words? 

The  subjects  were  also  divided  in  answering  this  question 
regarding  retention  of  the  words .  The  extremes  were  evidenced,  with 
some  subjects  stating  that  they  remembered  100%,  whereas  others  said 
they  remembered  almost  nothing.  With  few  exceptions,  all  agreed 
that  their  retention  was  strictly  short  term  in  nature.  A  recurring 
comment  was  that  an  additional  exposure  to  the  words  would  have  been 
helpful  in  committing  the  words  to  long-term  memory.  This  is 
interesting  from  the  standpoint  that  though  45  minutes  had  been 
allotted  for  the  study  of  the  vocabulary,  the  average  time  actually 
spent  on  task  was  roughly  twelve  minutes. 

Two  comments  made  by  separate  individuals  suggest  a 
potentially  significant  line  of  investigation.  The  first  commented 
that  "if  I  could  remember  them  initially,  then  it  was  no  problem, " 
and  another  stated  "some  words  seemed  to  be  easy  -  I  remembered  them 
long  term. "  Although  this  concept  is  certainly  not  new,  the  impact 
on  programs  for  learning  and  retention  is  that  the  process  must 
allow  for  students  to  select  out  the  words  that  they  know  and 
concentrate  on  the  words  they  find  to  be  more  difficult.  As 
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previously  mentioned,  some  degree  of  predictability  regarding 
difficulty  of  lexis  is  afforded  by  current  theory,  but  this  study 
suggests  a  noticeable  degree  of  individual  variation. 

Randomly  Selected  Case  Studies 

One  case  was  selected  randomly  from  each  of  the  four  groups 
for  a  more  in-depth  analysis.  Narrowing  the  focus  to  specific 
individuals  is  a  method  often  used  in  qualitative  studies  in  order 
to  delve  below  the  more  general  type  of  examination  that  normally 
accompanies  a  quantitative  study.  This  more  involved  study  helped 
bring  to  light  some  thoughts  and  factors  that  might  have  otherwise 
not  been  noticed,  or  might  have  received  less  emphasis. 

Case  #1 

This  case  was  drawn  at  random  from  the  group  that  received 
instruction  in  the  non-target  words  and  also  had  no  assistance  from 
the  on-line  dictionary  during  the  recall  protocols.  Specifically, 
this  case  reflects  Subject  #25  (ID:  GWI12153)  .  The  subject  was  able 
to  glean  and  remember  the  meaning  for  several  of  the  target  words. 

He  received  full  credit  for  two  words,  a  partial  understanding  of  a 
third  word,  and  familiarization  with  a  fourth.  These  meanings  were 
gained  from  incidental  learning,  which  took  place  during  the 
recalls,  that  being  the  only  exposure  to  the  words.  His  guesses  at 
words  whose  meanings  were  unknown  displayed  a  common  trait  observed 
in  a  majority  of  the  subjects'  responses,  the  tendency  to  use  the 
outward  form  and  match  it  against  an  English  word  in  an  attempt  to 
find  the  definition.  Examples  are  that  Raetsel  (puzzle)  was  thought 
to  be  rattle,  unnoetig  (unnecessary)  was  defined  as  unnoticeable, 
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and  Gelegenheit  (opportunity)  was  thought  to  have  something  to  do 
with  temperature.  This  type  of  aid  was  unavailable  for  the  words 
for  which  credit  was  given,  so  an  assumption  can  be  made  that  the 
meanings  were  actually  derived,  though  the  subject  did  not  believe 
that  he  knew  the  meanings  of  the  words.  This  was  unusual,  normally 
when  credit  was  given  the  subjects  indicated  that  they  had  some 
knowledge  of  the  credited  words.  The  immediate  level  of  knowledge 
score  was  9,  compared  with  a  delayed  score  of  5.  Understandably, 
the  subject  had  forgotten  the  meaning  of  one  of  the  words,  but 
interestingly  he  received  partial  credit  for  a  word  not  previously 
credited. 

The  measure  of  response  time  was  an  average  of  4.69  seconds 
for  the  immediate  test,  faster  than  the  group  average  of  5.75 
seconds,  and  6.27  seconds  for  the  delayed  test,  slower  than  the  5.87 
mean  achieved  by  the  group.  The  inconsistency  of  the  response  times 
for  this  individual  calls  into  question  the  efficacy  of  using  such 
response  times  as  measures  of  autoraaticity .  Although  seen  as  a 
separate  measure  of  lexical  knowledge,  rapid  response  times  should, 
in  general,  be  related  to  higher  levels  of  vocabulary  knowledge. 

This  is  not  the  case  with  this  individual  who  has  demonstrated  a 
very  poor  level  of  knowledge  concerning  the  vocabulary  in  question. 
The  fact  that  his  response  times  slowed  on  the  delayed  test  is  in 
agreement  with  theoretical  predictions.  The  size  of  the  change, 
however,  is  quite  large  and  places  him  well  above  the  group  mean, 
whereas  the  subject  was  well  below  the  group  times  on  the  immediate 
test. 

The  total  recall  score  for  this  individual  for  all  three 
articles  was  109,  considerably  higher  than  the  group  average  of 
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89.12.  Noteworthy  is  that  the  subject  had  one  of  the  top  overall 
recall  scores  for  the  travel  article. 

In  answer  to  the  questions,  the  subject  stated  that  he 
"enjoyed  participating  in  it,  but  I  wish  I  knew  what  its  aims  were.7' 
He  voiced  uncertainty  in  terms  of  the  quality  and  direction  of  the 
computer  programs  but  believed  that  the  words  learned  were  somewhat 
helpful  in  the  in  deriving  the  meaning  of  the  written  passages. 

Case  #2 

This  case  was  drawn  from  the  condition  where  the  subjects  were 
instructed  in  the  target  words  but  received  no  access  to  the 
dictionary  during  the  reading  of  the  passages.  The  case  represents 
Subject  #34  (ID:  GTR19991) .  This  subject  displayed  an  excellent 
recall  of  the  definitions  of  the  words  to  which  she  had  been 
exposed.  The  raters  gave  her  full  credit  for  thirteen  out  of 
eighteen  words  with  five  words  being  rated  as  unknown.  Those  words 
.  that  she  could  not  recall  she  still  recognized  having  seen  before. 
She  had  a  total  score  of  44,  which  was  quite  high  compared  with  the 
comparison  group  mean  of  32.04.  The  delayed  score  was  still  quite 
high  with  a  score  of  34  as  compared  to  the  group  mean  of  23.86.  The 
degree  of  reduction  in  level  of  knowledge  seems  to  parallel  the 
pattern  of  the  group,  which  indicated  approximately  an  eight  point 
decrease.  The  residual  level  is  still  quite  high  and  matches  her 
own  assessment  of  her  ability  to  retain  the  words. 

The  immediate  response  time  was  4.36  versus  the  group  average 
of  5.79,  and  her  delayed  score  of  6.73  was  well  above  the  average  of 
5.35.  Once  again,  the  subject  undergoes  a  tremendous  shift  in 
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response  time  despite  her  level  of  knowledge  showing  a  smaller 
decline. 

The  total  recall  score  was  also  above  the  group  average  of 
84.20  at  103.  This  subject  seems  to  have  been  able  to  make  good  use 
of  the  learned  material  in  getting  the  gist  of  the  article.  In 
addition,  as  witnessed  in  the  paragraph  below,  she  seems  have  been 
motivated  to  truly  applying  herself  to  the  task. 

This  subject  felt  that  the  study  was  a  good  one,  that  the 
computerized  instruction  had  helped  her  to  learn  the  words  well. 

She  was  surprised  at  how  well  she  retained  the  words,  and  believed 
that  the  learned  words  were  of  assistance  in  deciphering  the 
articles.  In  terms  of  difficulty  —  "It  was  difficult  but  it  made 
me  really  use  the  German  skills  that  I  have." 

Case  #3 

This  case  was  drawn  at  random  from  the  group  that  received 
instruction  in  non-targeted  words  and  had  lexical  access  during  the 
recall  protocols.  The  case  in  question  deals  with  Subject  #72  (ID: 
GPA12600) . 

The  level  of  knowledge  score  for  this  individual  was  rather 
low  despite  having  lexical  access  during  the  experiment.  None  of 
his  definitions  were  judged  as  full  credit  by  the  raters,  four 
received  partial  credit  and  the  remainder  of  the  points  came  from 
familiarity.  Further  investigation  showed  that  the  subject  made 
very  few  references  to  the  dictionary  function  during  the  recall 
process.  All  told,  he  looked  up  only  22  words  across  all  three 
protocols.  Only  one  of  the  words  for  which  he  received  partial 
credit  was  looked  up,  implying  that  the  other  meanings  were  either 
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gathered  incidentally  or  derived  in  some  other  manner.  Immediate 
word  knowledge  was  scored  a  ten,  slightly  below  the  group  average  of 
11.52.  Delayed  knowledge  was  rated  a  7,  well  below  the  average  of 
11.68. 

The  time  response  of  this  individual  is  largely  problematic  in 
as  much  as  many  of  his  response  times  exceeded  the  twenty  second 
limitation.  This  subject  was  one  of  the  worst  examples  of 
distraction  from  the  task  going  over  the  time  limit  nine  times. 

These  excessive  times  were  capped  at  20  seconds,  with  a  resulting 
average  time  of  12.42  seconds  versus  the  group  norm  of  7.30  seconds. 
This  problem,  though  exaggerated  in  this  circumstance,  once  again 
draws  attention  to  the  assumption  of  response  time.  Obviously,  some 
additional  controls  are  necessary  to  insure  that  response  times  are 
reflective  of  consideration  about  word  knowledge  and  not  some 
outside  distraction.  The  delayed  test  for  this  individual  had  no 
instances  of  going  over  the  limit  with  a  resultant  average  time  of 
7.42  seconds.  This  fit  closely  with  the  group  delayed  mean  of  7.19 
seconds. 

The  subject's  recall  score  of  97  closely  reflected  the  group 
average  of  102.  The  subject  seemed  to  spend  a  great  deal  of  time  on 
the  recalls  without  doing  very  much.  As  already  mentioned,  few 
words  were  referenced  and  the  length  of  the  recalls  was  quite  short. 
These  factors,  in  conjunction  with  the  numerous  times  that  he 
exceeded  the  time  limit  on  the  automaticity  measure,  suggest  that 
this  individual  appears  to  be  somewhat  easily  distracted  from  the 
task  at  hand. 

Despite  his  poor  performance,  he  enjoyed  the  study.  He  wrote 
that  he  actually  preferred  this  interactive  computer  process  to 
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learning  from  an  instructor,  citing  that  "I  do  not  feel  stupid  for 
asking  the  computer  for  a  word  I  feel  I  should  know."  He  felt  the 
study  was  too  difficult,  well  above  his  level  despite  the  fact  that 
he  is  an  intermediate  level  student.  He  repeats  his  comfort 
interacting  with  "an  indifferent,  impersonal  machine."  He  states 
that  he  "does  not  have  a  knack  for  remembering  things"  and  was 
unable  to  remember  learned  words  for  much  more  than  twelve  hours. 

Case  #4 

This  case  was  drawn  at  random  from  the  group  that  received 
instruction  in  both  the  targeted  vocabulary  as  well  as  access  to  the 
on-line  dictionary  during  the  recall  protocols.  Specifically,  the 
responses  of  Subject  #77  (ID:  GC018686)  were  examined. 

The  word  knowledge  for  this  individual  was  quite  good  with  a 
total  score  of  38  points,  above  the  group  average  of  32.40.  The 
delayed  word  knowledge  of  23,  on  the  other  hand,  was  below  the 
average  of  25.5.  This  subject,  though  learning  the  material  well 
initially,  exhibited  more  than  the  usual  amount  of  forgetting  over 
the  two  week  period  between  the  immediate  and  delayed  tests. 

In  terms  of  the  automaticity  measure,  this  subject  exhibited 
very  consistent  response  times.  His  average  response  times  of  6.5 
seconds  and  6.69  seconds  (delayed)  were  only  slightly  higher  than 
the  group  response  times  of  6.21  and  6.06  (delayed). 

The  subject's  recall  score  of  106  was  higher  than  the  group 
average  of  91.60.  In  analyzing  the  information  gathered  during  the 
recall  more  closely,  it  becomes  apparent  that  this  subject  put  good 
effort  into  attempting  to  understand  the  text.  He  spent  an  average 
of  11  minutes  and  30  seconds  on  each  protocol  (out  of  a  maximum 
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allowable  of  15  minutes)  ,  and  looked  up  54  words  across  all  three 
protocols. 

The  subject's  assessment  of  his  own  learning  and  retention 
were  good.  He  correctly  states  that  the  "vocabulary  was  easily 
learned,  but  I  didn't  retain  it  very  well".  This  fits  well  with  his 
level  of  knowledge  scores,  which  were  quite  high  initially,  but  then 
dropped  below  the  average  on  the  delayed  test.  He  also  echoes 
thoughts  about  the  difficulty  of  the  passages:  "It  was  difficult,  ■ 
almost  to  the  .point  where  I  stopped  trying."  When  queried  about 
this  method  of  learning  vocabulary,  the  subject  was  non-committal. 

He  qualified  his  statement  in  the  following  way,  "Reviewing  is  a 
necessity  if  the  vocabulary  program  is  to  be  effective.  Not  just 
reviewing  the  same  day,  but  on  separate  occasions."  This  theme  came 
up  on  other  responses  as  well,  and  fits  with  accepted  theory, 
particularly  that  of  Pimsleur  (1967),  who  proposed  and  strongly 
supports  the  concept  of  a  learning  and  reinforcement  schedule. 

Discussion 

Level  of  Word  Knowledge 

The  most  significant  result  pertaining  to  word  knowledge,  and 
numerically  the  most  significant  of  the  study,  was  the  enormous 
impact  of  prior  instruction  on  subsequent  word  knowledge.  Those 
subjects  who  had  been  instructed  in  the  definitions  of  the  target 
vocabulary  scored  well  above  the  non-focus  groups.  This 
superiority,  in  and  of  itself,  is  perhaps  not  too  surprising.  One 
might  expect  that  students  who  had  been  exposed  to  definitions 
should  be  more  able  to  provide  those  definitions  when  tested.  It  is 
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the  degree  of  difference ,  as  well  as  the  fact  that  even  the  delayed 
scores  of  the  focus  group  were  significantly  above  all  scores  by  the 
non-focus  group,  that  draw  attention.  The  MANCOVA  yielded  an 
F  ( 1 , 99 )  =  249.71  which  translates  to  a  p  =  .000.  It  had  been 
thought  that  the  advantage  of  instruction  would  be  offset  in  the 
other  groups  to  some  degree  by  lexical  access  and  the  learning  of 
definitions  through  contextual  clues. 

Although  lexical  access  did  not  initially  prove  to  be 
significant  at  the  .05  level  (p  =  .107),  subsequent  analysis  of  the 
scores  of  the  groups  not  instructed  in  the  target  words  indicated  a 
significant  advantage  in  level  of  word  knowledge.  This  interaction 
between  instruction  and  lexical  had  been  insignificant  in  the 
initial  analysis,  most  probably  because  the  effects  of  instruction 
on  the  targeted  words  was  so  overwhelming.  The  idea  that  word 
instruction,  subsequently  reinforced  through  lexical  access,  might 
bolster  overall  word  knowledge  above  that  from  instruction  alone  did 
not  turn  out  to  be  the  case. 

Part  of  the  problem  in  determining  the  effect  of  this  on-line 
access  is  that  the  usage  was  highly  variable  in  nature,  with  some 
subjects  looking  up  a  considerable  number  of  words  and  others 
relatively  few. 

Recall  Protocols 

Both  variables  had  a  non-significant  impact  on  the  overall 
recall  protocols.  Of  the  two  variables,  however,  lexical  access  did 
provide  the  higher  significance  (p  =  .123).  Despite  the  failure  to 
achieve  statistical  significance  at  the  .05  level,  this  is  another 
instance  where  the  potential  advantage  should  not  be  ignored.  As 
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mentioned  in  the  previous  section,  actual  usage  of  the  on-line 
definitions  was  highly  variable,  so  firm  conclusions  as  to  its 
effect  are  somewhat  problematic. 

The  distinction  between  focus  and  non-focus  groups,  or  lack 
thereof,  was  not  surprising.  The  main  difference  between  the  words 
was  that  the  non-focus  group  words  were  potentially  known  to  the 
students  prior  to  the  experiment  and  generally  deemed  not  as 
critical  (by  this  researcher)  to  the  overall  interpretation  of  the 
text.  Otherwise,  the  words  for  both  groups  were  all  derived  from 
the  recall  texts.  Whereas  the  overall  lack  of  significance  (p  = 
.238)  was  not  completely  unanticipated,  the  fact  that  the  advantage 
fell  to  the  non-focus  group  was  unexpected.  There  was  also  no 
significant  interaction  between  groups  and  lexical  access. 

The  results  from  this  section  serve  to  support  the  idea  that, 
though  the  connection  between  vocabulary  and  reading  comprehension 
exists,  the  exact  nature  of  the  relationship  is  complex  and 
dependent  upon  a  number  of  variables.  Specifically,  the  difficulty 
of  the  recalls  and  the  relative  density  of  unknown  words  made  it 
hard  to  achieve  significant  improvements  in  overall  recall  scores. 
The  difficulty  of  the  texts  can  be  attributed  to  some  degree  to  the 
fact  that  authentic  articles  were  utilized.  This  reflects  the 
current  push  in  the  language  teaching  field  towards  ever  greater 
utilization  of  these  types  of  authentic  materials .  Had  the  texts 
been  pedagogically  structured  with  fewer  unknown  words,  the  impact 
of  instruction  or  lexical  access  might  well  have  been  more  readily 
apparent . 
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Automaticity 


The  sole  significant  finding  for  automaticity  was  that  those 
without  lexical  access  had  significantly  faster  response  times  than 
those  with  lexical  access,  5.77  seconds  versus  6.75  seconds,  with  a 
resultant  £  =  .021.  In  comparing  the  focus  group  versus  the  non¬ 
focus  group,  the  focus  group  had  the  faster  response  (6.00  seconds 
vs  6.53  seconds),  but  the  difference  proved  to  be  non-significant. 
The  interaction  also  failed  to  reach  significance. 

The  fact  that  those  who  had  been  instructed  had  faster 
response  times  is  in  agreement  with  the  overall  concept  of 
automaticity  as  put  forward  here,  that  is,  that  those  with  a  better 
overall  concept  or  knowledge  of  the  word  should  be  able  to  access 
that  knowledge  more  rapidly.  Of  greater  interest,  however,  is  the 
fact  that  those  without  lexical  access  scored  significantly  quicker 
response  times  than  those  subjects  with  lexical  access.  This  is 
contrary  to  expectations  in  as  much  as  one  might  reasonably  expect 
those  with  lexical  access  to  have  at  least  a  slightly  better 
knowledge  of  the  target  vocabulary  than  those  without  such  access. 

The  measure  of  automaticity  was  the  most  problematic  area  of 
the  study.  In  addition  to  the  inconsistency  cited  above,  several 
other  concerns  were  evidenced.  First,  there  were  the  statistical 
outliers,  previously  mentioned,  who  had  excessive  times  that  were 
reduced  to  twenty  seconds.  Even  with  those  corrections,  however,  it 
is  likely  that  the  overall  impact  of  those  subjects  negatively 
affected  the  validity  and  reliability  of  the  automaticity  scores. 
Second,  the  overall  pattern  of  response  times  is  erratic  and 
inconsistent  among  individuals,  yet  when  examined  by  groups  showed 
little  deviation  from  the  overall  mean.  Lastly,  the  delayed  scores 
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exhibited  faster  response  times  than  the  immediate  scores.  This 
would  also  be  contrary  to  expectations  if  the  measures  were  working 
as  desired,  because  if  some  degree  of  knowledge  is  lost  one  would 
also  await  a  slight  loss  in  automaticity . 

It  seems  clear  that  variables  other  than  automaticity  came 
into  play  and  that  they  affected  the  response  times  of  the  students. 
One  factor  was  attention  to  the  task,  which  was  not  always  as  sharp 
as  it  should  have  been.  Another  factor  seemed  to  be  what  might  be 
titled  a  pattern  of  expectation.  That  is,  that  those  students  who 
had  little  or  no  knowledge  of  the  words  came  to  expect  and 
anticipate  that  their  response  to  each  word  would  probably  be  that 
it  was  "Unknown."  In  contrast,  those  who  had  some  sort  of  exposure 
might  be  expected  to  have  to  wrestle  with  each  word  at  least 
slightly.  The  anecdotal  evidence  to  this  pattern  of  expectation  is 
that  some  of  the  subjects  with  low  knowledge  scores  had  extremely 
rapid  response  times.  This  pattern  might  also  explain  why  the 
delayed  scores  indicated  greater  automaticity  than  the  immediate 
scores . 

Reading  Time 

An  investigation  of  time  spent  reading  the  recalls  was  made 
after  the  fact,  and  revealed  that  those  who  had  lexical  access  spent 
approximately  43%  more  time  on  task  than  those  students  who  had  no 
access  to  the  on-line  definitions.  This  figure  is  in  complete 
agreement  with  prior  research,  which  generally  places  the  increase 
in  reading  time  due  to  dictionary  access  at  between  40%  and  50%. 
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Qualitative  Aspects  of  the  Study 


The  general  impression  was  that  the  study  and  the  associated 
computer  programs  were  well  organized,  and  that  the  process  ran 
smoothly.  Although  it .might  be  an  overstatement  to  say  that  the 
cadets  enjoyed  taking  part,  they,  by  and  large,  found  it  interesting 
and  did  not  regret  being  involved  in  the  study.  As  has  been  the 
case  in  virtually  all  of  the  recent  research,  the  students  were 
completely  comfortable  interacting  with  computers  during  the  study. 

Specifically,  the  cadets  seemed  to  appreciate  the  computer 
program  that  instructed  them  in  the  definitions.  This  program 
worked  them  through  several  cycles  of  exposure,  and  then  let  them 
proceed  at  their  own  pace.  The  instruction  of  vocabulary  seems  to 
be  something  that  is  particularly  well  suited  for  the  computer.  A 
theme  evidenced  repeatedly  throughout  the  comments  was  that  the 
instruction  would  have  been  enhanced  had  there  been  more 
opportunities  for  exposure  to  the  words  (more  cycles  at  different 
times) .  Given  that  most  of  the  subjects  utilized  only  a  portion  of 
the  45  minutes  available  to  study  the  vocabulary,  it  seems  that 
concentrated  study  needs  cyclical  reinforcement.  The  cadets  were 
not  as  enthusiastic  about  the  evaluation  programs,  though  from  the 
written  responses,  it  was  apparent  that  at  least  some  of  the 
negative  feelings  came  from  the  degree  of  difficulty  of  the  passages 
and/or  the  lack  of  knowledge  of  the  vocabulary. 
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CHAPTER  5 


SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

Overview  of  the  Study 

The  purpose  of  the  study  was  to  investigate  whether  there  were 
benefits  to  be  derived  from  the  systematic  instruction  of  vocabulary 
above  and  beyond  those  available  from  either  simple  exposure,  to  the 
words  in  context,  normally  referred  to  as  incidental  learning,  or 
exposure  combined  with  lexical  access.  One  hundred  and  thirty 
college  level  beginning  and  intermediate  German  students  were 
randomly  placed  into  one  of  four  different  groups.  These  groups 
were  defined  by  two  variables:  on-line  dictionary  access  or  no 
access  and  whether  or  not  they  received  instruction  in  the  target 
vocabulary.  One  hundred  subjects  were  randomly  selected  for  the 
final  analysis. 

A  vocabulary  checklist  test  was  administered  to  insure  that 
none  of  the  subjects  had  prior  knowledge  of  the  target  words.  The 
students  were  also  familiarized  with  the  procedure  for  recall 
protocols,  as  well  as  the  computer  programs  that  they  would  be 
using. 

The  first  day  of  the  process  was  familiarization  with  the 
computer  programs.  The  second  day  of  the  testing  cycle  was 
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computerized  vocabulary  instruction.  Half  of  the  subjects  learned 
the  definitions  for  the  target  vocabulary,  and  the  remainder 
received  instruction  in  non-target  vocabulary  derived  from  one  of 
the  protocols.  On  the  third  day,  all  the  subjects  viewed  three 
passages  on  the  computer  and  were  asked  to  provide  recalls.  Again, 
50%  of  the  subjects  had  access  to  on-line  dictionaries  during  these 
recalls,  and  the  remainder  had  no  assistance.  The  fourth  and  final 
day  of  the  testing  cycle  was  a  supply  definition  test  of  the  target 
vocabulary.  The  subjects  were  also  asked  questions  about  several 
qualitative  aspects  of  the  study.  A  delayed  supply  definition  test 
was  given  two  weeks  later. 

Once  the  data  had  been  averaged  and  sorted,  the  necessary 
analyses  were  conducted  and  the  results  tabulated.  Qualitative 
aspects  were  also  examined  in  the  hopes  that  a  deeper  insight  might 
be  provided. 

Summary  of  Findings  and  Conclusions 

Finding  #1: 

The  strongest  finding  of  the  study  was  that  those  students  who 
were  instructed  in  the  targeted  vocabulary  were  better  able  to 
produce  an  acceptable  definition  when  called  upon.  The  significance 
(jd  <  .001)  indicates  a  very  powerful  effect  for  instruction  on 
subsequent  word  knowledge.  Even  the  delayed  scores  by  the  focus 
group  were  well  above  the  immediate  scores  by  those  subjects  who 
were  not  instructed. 

These  findings  are  in  keeping  with  the  results  of  the  meta¬ 
analyses  conducted  by  Stahl  and  Fairbanks  (1986)  and  Mezynski 
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(1983).  These  analyses  indicated  positive  results  for  all  forms  of 
vocabulary  instruction,  and  statistically  significant  results  for 
many  of  them.  More  recently,  Coady's  (1993)  study  utilizing 
computerized  vocabulary  training  resulted  in  findings  largely 
supporting  the  favorable  impact  of  direct  vocabulary  instruction. 
Coady  used  more  programmatic  instruction  of  many  words  over  a  longer 
period  of  time,  but  the  outcomes  were  very  positive  in  terms  of  the 
effectiveness  of  direct  instruction.  Specifically,  Coady 
demonstrated  that  computerized  instruction  could  give  significant 
advantage  in  terms  of  subsequent  vocabulary  knowledge  (p  <  .01) 

Another  notable  point,  mentioned  above,  is  the  resilience  of 
the  instructed  words  in  memory.  The  overall  level  of  knowledge  for 
the  instructed  students  dropped  from  a  score  of  32.22  to  24.68,  or  a 
drop  of  23%.  This  drop  over  the  course  of  two  weeks,  despite  being 
statistically  significant,  is  quite  promising  in  as  much  as  it  is 
less  than  prior  research  indicates  as  the  normal  amount  of 
forgetting.  Even  this  delayed  score  is  significantly  higher  than 
the  immediate  score  of  9.38  for  the  subjects  not  instructed  in  the 
target  words . 

Finding  #2 

Another  finding  worthy  of  attention  is  the  short  period  of 
time  that  the  subjects  utilized  in  studying  the  words.  In  contrast 
to  many  previous  studies  where  instruction  time  was  forced  upon 
students,  and  was  often  quite  lengthy  in  terms  of  time  spent  per 
word,  this  study  allowed  the  students  to  select  their  own  pace  in 
reviewing  the  words.  Although  they  had  up  to  45  minutes  to  use  the 
vocabulary  instruction  program,  most  students  were  done  by  roughly 
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twelve  minutes,  with  very  few  exceeding  fifteen  minutes.  (These 
times  are  rough  estimates  by  the  researcher,  the  timing  function  of 
the  software  did  not  function  properly).  This  calculates,  on 
average,  to  less  than  a  minute  spent  with  each  word.  Once  again, 
this  pace  of  learning  for  adult  learners  agrees  with  much  of  the 
extant  research  wherein  learners  are  allowed  to  choose  the  pace  of 
their  learning.  In  a  study  by  Wheatley,  Muller,  &  Miller  (1993), 
they  found  that  their  students  spent  an  average  of  one  minute  on 
each  word,  very  close  to  the  time  spent  by  learners  in  the  current 
study. 

Finding  #3 : 

The  availability  of  an  on-line  dictionary,  though  providing  a 
higher  overall  level  of  knowledge  scores,  failed  to  register  as 
statistically  significant  in  the  original  MANCOVA.  Upon  further 
analysis,  however,  it  was  demonstrated  that  if  one  isolates  the 
groups  that  were  not  instructed  in  the  target  words,  the  effect  of 
lexical  access  is  dramatic  (p  =  .004).  This  interaction  will  be 
more  fully  discussed  under  finding  #6.  This  benefit  of  lexical 
access  is  consistent  with  prior  research.  Knight  (1992)  found  that 
those  subjects  with  dictionary  access  achieved  a  mean  score  of  4.95 
while  those  without  access  had  a  mean  score  of  1.72  (p  <  .001). 
There  are  other  striking  similarities  when  one  compares  the  current 
study  with  the  high  verbal  population  from  Knight's  study.  The  no 
dictionary  group  in  Knight's  study  was  able  to  learn  7%  of  the 
vocabulary,  whereas  the  group  with  dictionary  access  learned  21%. 

In  this  study,  the  group  without  lexical  access  averaged  13%  of  the 
vocabulary  knowledge  points  available,  while  the  group  with  lexical 


91 


access  averaged  23%.  Both  studies  indicate  an  approximate  two  to 
three-fold  increase  of  the  vocabulary  knowledge  with  the  addition  of 
lexical  access.  Luppescu  and  Day's  research  also  supports  this 
claim.  In  Luppescu  and  Day  (1993)  ,  they  found  that  the  mean 
measures  of  vocabulary  knowledge  for  the  group  using  dictionaries 
was  about  50%  greater  than  those  of  the  group  without  dictionaries. 

One  factor  that  may  have  detracted  from  an  even  greater 
increase  in  performance  tied  to  lexical  access  is  that  the  method 
used  in  the  study  for  accessing  word  definitions  was  less  convenient 
than  the  manner  the  cadets  typically  use.  If  they  desired  to  see  a 
definition  during  the  protocols,  the  cadets  were  required  to  scroll 
through  a  list  to  select  the  appropriate  word.  They  could  then 
click  on  the  word  to  receive  a  brief  definition.  This  method, 
though  close  in  nature  to  actual  dictionary  use,  was  more  awkward 
than  the  cadet's  normal  practice  of  clicking  on  the  word  in  the  text 
to  get  a  definition.  This  was,  in  all  likelihood,  largely 
responsible  for  the  varying  degree  of  words  looked  up,  with  some 
subjects  making  much  greater  use  of  the  lexical  access  feature  than 
other  students  did. 

In  order  to  check  for  the  possible  impact  of  number  of  words 
looked  up  and  vocabulary  knowledge  an  analysis  was  conducted  of  the 
correlations  between  total  number  of  words  checked  and  knowledge 
scores,  as  well  as  between  number  of  target  words  looked  up  and 
knowledge  scores.  The  resulting  correlations  were  quite  low  (total 
words:  .131;  target  words:  -.057)  suggesting  that  the  situation 
cannot  be  reduced  to  simply  the  number  of  words  examined. 
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Finding  4 : 


The  recall  protocol  scores  were  largely  unaffected  by  either 
variable.  Lexical  access,  however,  did  appear  to  be  helpful  at  just 
slightly  below  the  significant  level  (p  =  .123).  This  is  in  keeping 
with  the  research  to  date,  which  generally  notes  some  degree  of 
advantage  to  protocol  scores  with  lexical  access.  Many  of  the 
results  lay  either  just  above  or  just  below  the  point  of  statistical 
significance.  Two  separate  meta-analyses,  Mezynski  (1983)  and  Stahl 
and  Fairbanks  (1986),  examined  past  studies  on  the  effect  of 
vocabulary  instruction  and  subsequent  text  comprehension.  Both 
analyses  concluded  that  those  studies  that  had  shown  significant 
improvement  in  reading  comprehension  utilized  instruction  that 
provided  repeated  exposures  to  the  words,  presented  both 
definitional  and  contextual  information,  and  encouraged  active 
processing.  When  taken  as  a  whole,  the  studies  to  date  suggest 
that,  "in  order  to  affect  comprehension,  instruction  may  need  to  go 
beyond  simply  establishing  accuracy  of  associations  between  words 
and  definitions"  (Beck  and  McKeown,  1991,  page  806) . 

In  contrast,  the  research  of  Knight  (1992),  despite  providing 
only  associative  connections,  suggested  that  lexical  access  greatly 
enhanced  subsequent  comprehension  (p  <  .001).  One  mitigating  factor 
in  the  current  study  that  may  help  to  explain  the  lack  of 
significance  is  the  fact  that  all  groups  had  either  instruction  on 
some  of  the  words  or  lexical  access.  Had  there  been  groups  that  had 
no  prior  exposure  to  any  of  the  words,  as  was  the  case  in  the  Knight 
experiment,  it  is  likely  that  there  would  have  been  greater 
improvements  due  to  lexical  access,  perhaps  adequate  to  lead  to 
significance.  Overall,  the  results  of  the  study  are  consistent  with 
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prior  research,  and  suggest  the  even  associative  instruction 
provides  some  benefit  in  subsequent  comprehension,  though  chances  of 
significant  improvement  increases  with  greater  exposure  in  varied 
contexts  and  activities  requiring  deeper  processing. 

The  fact  the  non- focus  group  outscored  the  focus  group,  upon 
reflection,  is  not  that  surprising.  Both  sets  of  words  were 
selected  from  the  texts  as  being  useful  in  accessing  the  gist  of  the 
materials.  In  the  case  of  the  focus  group,  the  words  selected  were 
restricted  to  the  words  not  known  by  any  of  the  subjects  of  the 
research.  Although  the  words  for  the  focus  group  were  selected  more 
deliberately,  the  words  chosen  for  the  non-focus  group  appear,  in 
the  end,  to  have  been  more  helpful  to  the  overall  recalls. 

Finding  #5: 

The  measures  of  automaticity,  acknowledged  in  advance  as  the 
most  problematic  aspect  of  the  study,  provided  a  confusing  mixture 
of  results.  The  sole  significant  finding  was  that  those  without 
lexical  access  had  significantly  faster  response  times  than  those 
without  lexical  access  (p  =  .021).  Because  of  a  number  of  factors, 
it  is  highly  questionable  whether  or  not  the  scores  can  be  properly 
relied  upon  to  answer  the  original  questions  regarding  automaticity. 

Specifically,  the  following  problem  areas  lead  one  to  ask 
about  the  validity  of  using  the  response  times  from  this  research  as 
measures  of  automaticity.  First,  there  exists  the  problem  of  the 
statistical  outliers,  those  individuals  who  had  response  times  so 
lengthy  that  they  required  correction.  Even  after  these  excess 
times  had  been  capped  at  twenty  seconds,  the  cumulative  times  for 
some  of  these  individuals  skewed  the  time  results.  Another  problem 
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was  the  erratic  nature  of  the  results,  which  presented  many  scores 
that  run  counter  to  the  theory  of  automaticity  that  is  being 
examined.  By  way  of  example,  as  mentioned  above,  those  subjects 
without  lexical  access  indicated  a  faster  response  than  those 
subjects  with  lexical  access,  and  almost  all  the  delayed  scores  were 
faster  than  the  immediate.  Both  of  these  results  are  contrary  to 
the  theory  that  greater  (or  more  recent)  exposure  should  decrease 
response  times.  A  final  problem  area,  to  some  extent  exacerbated  by 
the  first,  is  the  generally  long  response  times  when  compared  with 
Gahren ' s  study  (1993).  Gahren  had  a  mean  response  time  across  all 
categories  of  2.59  seconds  compared  with  this  study's  average  of 
6.19  seconds.  The  question  then  is  why  these  measures  did  not 
appear  to  function  as  desired  in  the  current  research. 

In  Gahren  (1993)  ,  upon  which  the  measures  of  automaticity  used 
in  this  experiment  were  based,  the  researcher  was  dealing  with  a 
select  group  of  advanced  language  learners  who  had  greater  long-term 
exposure  to  many  of  the  words.  These  factors,  in  combination  with 
others,  such  as  a  higher  level  of  motivation  and  the  ability  to  test 
themselves  at  their  own  convenience,  possibly  served  to  ensure  more 
appropriate  response  times  in  Gahren 's  study. 

Finding  #6: 

The  original  MANCOVA  exhibited  no  significant  interactions, 
but  subsequent  analysis  showed  a  significant  effect  for  lexical 
access  in  the  non-focus  groups.  Previously  discussed  on  several 
occasions,  there  was,  in  fact,  interaction  between  the  factors  of 
group  and  lexical  access  on  the  variable  of  knowledge,  but  that  this 
was  masked  by  the  tremendous  influence  of  instruction  on  the 
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targeted  vocabulary.  Further  impeding  the  ability  of  the  MANCOVA  to 
detect  significant  interactions  was  the  relatively  low  observed 
power  for  all  three  of  the  univariate  interaction  analyses.  For  the 
dependent  variable  of  knowledge  mentioned  above,  the  observed  power 
was  .275,  meaning  that  there  was  roughly  a  70%  chance  of  missing  a 
significant  interaction,  even  if  it  was  actually  present.  The  two 
other  interactions  for  response  times  and  recall  scores  had  even 
lower  observed  powers,  .262  and  .068  respectively,  indicating  an 
even  greater  chance  of  missing  significant  interactions.  In  should 
be  noted  that  in  Figures  3,  4  and  5,  which  graph  the  interactions, 
the  common  tendency  is  for  the  lines  to  converge.  Perhaps  with  a 
better  design  and  stronger  power,  existing  interactions  would  stand 
a  greater  chance  of  being  detected. 

It  was  thought  that  the  interaction  of  prior  instruction  and 
subsequent  exposure  to  the  words  in  context  might  prove  to  be 
symbiotic.  The  results  showed  that  this  seemed  not  to  be  the  case, 
with  all  of  the  analyses  remaining  well  removed  from  significance. 

It  had  been  theorized  that  through  a  greater  number  of  exposures,  in 
particular  exposure  in  context  following  direct  instruction  in 
definitions  might  provide  fertile  ground  for  greater  gains  in  all 
three  dependent  variables.  These  expectations  were  suggested  by  the 
research  of  McKeown  (1993)  in  which  she  forwarded  the  belief  that 
prior  instruction  provides  an  "initiating  event,"  or  a  rough 
acquaintance  with  the  word.  This  same  line  of  thought  was  evident 
in  Graves  and  Penn  (1986)  wherein  they  suggest  that  a  degree  of 
knowledge  of  certain  words  might  prevent  the  learner  from  stumbling 
over  those  words,  and  allow  greater  concentration  on  the  remainder 
of  the  text. 
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The  lack  of  interaction  may  well  be  an  accurate  reflection  of 
the  state  of  affairs,  but  several  additional  mitigating  factors  must 
be  considered.  The  general  difficulty  of  the  passage,  cited  often, 
is  one  concern.  Another  element  is  the  relative  density  of  unknown 
words  in  the  text,  which  is,  in  this  case,  fairly  high.  These  two 
factors  combine  to  make  it  somewhat  difficult  to  use  contextual 
clues  to  discern  word  meaning. 

Finding  #7 : 

The  verbal  skills  of  these  subjects  had  virtually  no  impact  on 
how  they  responded  to  variables  of  instruction  and  lexical  access. 
The  MANCOVA  suggests  an  almost  absolute  lack  of  significance  for 
this  factor.  .  For  those  subjects  with  ACT  scores,  the  reading  score 
was  then  examined  against  the  recalls,  whereas  the  English 
(vocabulary  score)  was  run  against  word  knowledge.  In  every  case, 
the  results  proved  insignificant. 

The  most  likely  explanation  for  these  results  is  attenuation. 
Despite  differences  in  scores,  the  students  as  a  whole,  when 
compared  against  previous  research,  would  classify  as  high  to  very 
high  verbal  ability.  Thus,  although  individual  cases  may  indicate 
superior  performance  due  to  ability,  this  was  not  reflected  for  the 
entire  group  as  might  have  been  the  case  had  there  been  the  range  of 
verbal  abilities  reflected  in  previous  research. 

Finding  #6: 

Reading  time  for  those  students  with  lexical  access  was  43% 
greater  than  for  those  subjects  without  lexical  access.  This  is  in 
keeping  with  prior  research,  which  consistently  places  the  time 
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increase  in  the  40%  to  50%  range.  Specifically/  Knight  (92)  found 
that  her  high  verbal  group  required  41%  more  time,  and  the  low 
verbal  group  needed  44%  more  time. 

Finding  #9: 

Students  were  very  comfortable  interacting  in  a  computer 
environment.  The  subjects  seem  to  enjoy  the  control  the  computer 
interface  afforded  allowing  them  to  proceed  at  their  own  pace. 
Several  students  also  commented  on  the  fact  that  there  was  a  certain 
amount  of  comfort  in  the  fact  that  the  computers  were  non- 
judgmental.  Again,  this  affinity  for  the  computer  environment  only 
confirms  what  much  previous  research  has  already  shown,  specifically 
that  such  computerized  instruction  enhances  student  achievement, 
involvment  and  motivation  (Wheatley,  Muller,  &  Miller,  1993). 

A 

Notably,  the  computerized  instruction  used  by  Coady  (1993)  in  his 
first  experiment  proved  so  beneficial  and  popular  that  "it  did  not 
seem  ethical  to  deny  the  students  access  to  these  materials,"  and 
the  replication  was  carried  out  without  a  control  group. 

The  factors  favoring  computers  are  perhaps  more  potent  at  the 
Air  Force  Academy  than  many  other  places.  The  students  all  own 
their  own  computers,  which  they  are  required  to  use  for  most 
assignments,  and  are  intimately  familiar  with  going  on-line  for 
necessary  information  and  communication.  The  computerized  language 
lab  where  the  research  took  place  is  an  environment  with  which  they 
are  completely  familiar,  in  so  far  as  they  meet  in  the  lab  every 
other  day. 
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Finding  #10: 


Many  subjects  cited  the  fact  that  some  words  seemed  to  be 
learned  much  more  easily,  and  that  those  same  words  were  more  easily 
retained.  These  comments  serve  to  highlight  one  of  the  factors  that 
make  computerized  vocabulary  instruction  such  a  potentially  powerful 
tool  -  namely  the  ability  for  the  student  to  choose  to  spend  more 
time  on  those  words  they  find  difficult,  and  to  control  the  degree 
of  exposure  to  the  words 

Finding  #11; 

The  instruction  of  vocabulary  seems  particularly  well  suited 
to  the  computer.  The  most  positive  comments  made  about  the  research 
had  to  do  with  the  vocabulary  instruction  program.  Although  this 
program  was  little  more  than  an  elementary  flash  card  program,  the 
students  were  virtually  unanimous  in  voicing  their  favor.  As 
mentioned,  the  response  to  Coady's  program,  which  was  more  elaborate 
(in  the  sense  that  it  was  more  than  mere  association)  ,  as  well  as 
more  long-term,  was  even  more  positive. 

The  allure  of  such  instruction  is  readily  apparent.  The 
students  have  complete  control  over  their  learning  in  the  sense  that 
they  can  choose  which  words  need  more  work,  and  also  set  their  own 
pace  for  progressing  through  the  words.  This  study  has  also 
demonstrated  that  such  instruction  can  produce  noticeable  results 
with  relatively  little  time  expenditure  --  in  this  case  an  average 
of  less  than  one  minute  per  word.  Clearly,  more  elaborate 
programming  requiring  deeper  processing  and  more  targeted  feedback 
would  likely  yield  greater  improvement,  but  the  point  to  be  made  is 
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that  even  basic  associative  word  instruction  is  well  received  and 
capable  of  providing  significant  results. 

Finding  #12: 

When  forced  to  guess  at  the  meaning  of  an  unknown  word  the 
students  normally  attempted  to  match  the  outward  form  with  a  known 
English  word.  They  would  guess  that  Ratsel  (puzzle)  meant  rattle  or 
that  Gelegenheit  (opportunity)  had  something  to  with  temperature. 

The  fact  that  the  cadets  would  seek  to  make  these  types  of 
.connections  is  both  intuitive,  and  in  agreement  with  the  factors 
influencing  the  difficulty  of  learning  new  words,  a  key  factor  being 
similarity  to  known  words  in  the  student's  Ll  lexis. 

Finding  #13: 

Subjects  were  generally  well  aware  of  their  level  of  word 
knowledge.  The  instances  of  students  believing  they  knew  a  word  and 
supplying  an  incorrect  meaning,  or  thinking  they  did  not  know  a  word 
and  arriving  at  a  correct  definition  were  virtually  non-existent. 
There  were  some  cases  where  subjects  believed  they  had  partial 
knowledge  that  proved  not  to  be  the  case,  but  even  those  instances 
were  few  and  far  between. 

Finding  #14: 

The  computerized  recall  protocol  procedure  worked  well  and 
appears  to  have  resulted  in  a  good  overall  assessment  of  the 
subjects'  reading  comprehension.  As  mentioned  earlier,  many  of  the 
cadets  thought  the  recall  process  was  a  good  one,  and  even  those 
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subjects  who  did  not  like  the  process  believed  that  it  would 
accurately  reflect  their  level  of  comprehension. 

As  a  rough  check,  an  examination  was  made  between  the  means  of 
the  scores  produced  by  this  current  test  group  versus  the  scores 
produced  by  the  level  one  subjects  in  Dr.  Heinz's  study.  The  level 
one  group  included  a  group  of  students  who  were  more  advanced  than 
the  subjects  in  the  current  study,  but  that  group  received  no  prior 
instruction  or  lexical  access.  One  might  expect  these  two  groups  to 
produce  similar  scores  on  the  recalls,  which,  as  seen  below  in  Table 
9,  was  precisely  the  case. 


Current  Study 

Heinz  Level 

JL 

Batman  Article 

35.39 

36.32 

Bernhardt  Letter 

27.92 

27.87 

Travel  Article 

29.56 

33.28 

Table  9 

A  Comparison  of  the  Raw  Protocol  Scores 

between 

the  Current 

Study 

and  the  Level  One 

Subjects  in  the  Heinz 

Study 
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Conclusions 


The  study  has  demonstrated  that  direct  vocabulary  instruction, 
even  in  its  most  basic  form,  can  further  vocabulary  learning  above 
the  levels  available  through  either  pure  incidental  learning,  or 
incidental  learning  and  lexical  access  in  combination.  The  degree 
to  which  vocabulary  instruction  can  improve  the  level  of  vocabulary 
knowledge  will  vary,  but  this  study,  along  with  some  of  the  more 
recent  research,  helps  illustrate  the  importance  of  returning  the 
instruction  of  vocabulary  to  it's  proper  place  in  the  foreign 
language  curricula.  In  particular,  good  computerized  instruction 
programs  can  provide  students  with  control  over  the  selection  and 
pacing  of  their  learning,  ample  "depth  of  processing,"  and  a  means 
by  which  to  assess  their  vocabulary  knowledge  and  progress. 
Computerized  instruction  can  accomplish  all  of  the  above  and  appears 
to  be  able  to  do  so  in  an  efficient  manner,  thereby  dispelling  some 
of  the  claims  that  direct  instruction  is,  of  necessity,  too  time 
intensive  to  be  useful. 

In  the  absence  of  instruction,  the  addition  of  lexical  access, 
though  increasing  reading  time  by  roughly  half,  is  capable  of 
bringing  about  a  significant  improvement  in  the  level  of  vocabulary 
knowledge.  Lexical  access  also  assisted  the  performance  of  subjects 
on  subsequent  reading  comprehension  tasks,  but  this  increase 
remained  shy  of  statistical  significance.  These  findings  concerning 
lexical  access  provide  support  to  many  of  the  conclusions  arrived  at 
by  Knight  (1992)  . 

The  issue  of  automaticity,  always  somewhat  problematic, 
remains  so.  The  response  times  provided  by  this  study  were  so 
diverse  in  nature,  and  differed  to  such  an  extent  from  previous 
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research,  as  well  as  from  theoretical  expectations,  that  one  is 
precluded  from  drawing  any  firm  conclusions.  In  fact,  examining  the 
factors  that  made  the  issue  so  difficult  for  this  study  begs  the 
question  as  to  what  degree  such  factors  may  have  impacted,  albeit  to 
a  much  lesser  degree,  prior  studies. 

The  use  of  computerized  recall  protocols  worked  smoothly  and 
efficiently,  and  a  comparison  of  the  results  of  this  study  with  the 
results  from  an  equivalent  group  in  Dr.  Heinz's  original  study  speak 
to  the  reliability  of  this  procedure.  Although  the  program  may  need 
some  refinements,  the  current  study  lends  support  to  the  viability 
of  computerized  recalls,  and  has  once  again  demonstrated  the 
practicality  of  using  the  procedure  on  large  groups  of  students. 

In  keeping  with  most  of  the  recent  research,  the  students  were 
comfortable  interacting  with  the  programs,  as  well  as  with  the 
general  computer  environment.  Many  were  enamored  with  the 
vocabulary  instruction  program,  despite  the  fact  that  it  was 
extremely  rudimentary.  In  terms  of  learning  the  vocabulary,  the 
students  found  that  some  vocabulary  were  easier  to  learn  and 
remember  than  other  words.  When  a  meaning  was  unknown,  the  students 
almost  always  attempted  to  match  the  outward  form  to  the  closest 
English  equivalent.  Lastly,  the  cadets  were  capable  of  accurately 
determining  their  own  level  of  word  knowledge. 

Pedagogical  Implications 

The  pedagogical  implications  are  clear,  the  computer  is 
finally  coming  into  its'  own  as  a  tool  in  the  instruction  of 
vocabulary.  Even  the  most  basic  of  flash-card  type  programs  is 
capable  of  providing  significant  gains  in  vocabulary  knowledge,  in 
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addition  to  being  well  received  by  the  students.  More  elaborate 
programs  that  provide  a  number  of  different  exposures  to  words  in 
varying  contexts,  allowing  for  greater  depth  of  processing,  should 
prove  to  be  even  more  powerful.  There  are  currently  several  efforts 
underway  at  the  Air  Force  Academy  to  put  such  programs  on  the 
intraweb.  The  German  syllabi  are  currently  on-line,  and  in  a  recent 
survey  (not  related  to  this  study)  the  students  indicated  a  strong 
desire  to  have  access  to  a  vocabulary  program  that  would  allow  them 
to  learn  the  words  and  evaluate  their  knowledge.  The  primary  effort 
at  the  moment  would  allow  instructors  to  enter  vocabulary  into 
databases  and  then  to  put  the  programs  on  the  web.  This  programming 
should  be  complete  by  the  summer. 

This  study  also  supports  the  idea  of  lexical  access  whenever 
possible.  There  existed  a  belief  in  the  field  of  foreign  language 
education  that  students  should  discern  meaning  from  context.  This 
approach  has  been  problematic,  particularly  from  the  standpoint  that 
the  student  is  unable  to  ascertain  whether  or  not  his  "guess"  is 
correct.  At  the  Academy,  the  students  have  full  lexical  access  when 
they  are  viewing  video  materials  in  the  Language  Learning  Center. 
They  are,  however,  encouraged  to  try  and  derive  meaning  from  context 
before  resorting  to  looking  up  words.  The  next  pedagogical  step  is 
to  place  reading  and  listening  comprehension  tasks  on-line,  or  on 
CD,  that  also  provide  them  the  capability  for  word  look-up. 

Dr.  Heinz's  procedure  for  testing  reading  comprehension  is 
laden  with  potential.  What  it  would  require  for  wider 
implementation  is  a  front-end  template  that  would  allow  instructors 
to  enter  text,  the  values  of  the  propositions,  and  to  establish 
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links  within  the  text.  This  process  is  currently  very  labor  and 
programming  intensive. 

There  appear  to  be  two  primary  advantages  to  computerized 
instruction  that  apply  specifically  to  vocabulary  instruction,  but 
also,  in  a  more  general  way,  to  all  instruction.  The  first 
advantage  is  that  the  student  has  control  over  his  own  learning. 
Given  the  diverse  backgrounds  and  abilities  in  any  foreign  language 
learning  population,  it  is  necessary  to  provide  different  paths  to 
higher-level  vocabulary  knowledge.  The  second  advantage,  as  yet  to 
be  harnessed,  is  the  assessment  and  feedback  the  computer  can 
provide  to  both  the  student  and  instructor. 

Recommendations 

Several  aspects  of  the  current  study  require  further  research 
and  elaboration.  Perhaps  most  importantly,  the  results  require 
substantiation  in  the  form  of  future  studies  that  alter  some  of  the 
variables  attendant  to  this  research.  The  words  selected,  the 
density  of  the  unknown  words  within  a  given  text,  and  the  context  in 
which  these  words  are  viewed  may  all  have  considerable  impact  upon 
the  nature  of  the  results.  If,  after  varying  these  factors,  the 
results  remain  consistent  with  those  found  in  this  study,  then 
firmer  conclusions  may  be  possible. 

A  point  of  weakness  in  the  current  study  is  the  observed  power 
for  the  covariant  of  verbal  skill  and  the  interactions.  Future 
research  should  be  designed  to  provide  greater  power  when  examining 
these  aspects . 

Another  requirement  for  future  will  be  the  examination  of  a 
wider  array  of  resultant  data  about  the  level  of  word  knowledge. 
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This  study  used  only  a  supply  definition  format.  Using  multiple 
choice  and  other  identification  tasks  would  lend  depth  to  the  data. 
In  past  research,  such  additional  dependent  variables  have  tended  to 
mirror  the  supply  definition  results,  but  generally  indicate  higher 
overall  levels  of  word  knowledge. 

Additional  studies  are  also  required  to  determine  if  the 
pattern  of  results  found  change  when  tested  across  the  full  range  of 
verbal  ability.  The  restriction  to  high  verbal  ability  populations 
is  an  admitted  limitation  of  this  study. 

Lastly,  this  research  should  ideally  be  integrated  into  the 
student's  normal  flow  of  instruction.  The  evaluation  tasks  should 
be  an  integrated  segment  of  the  curricula,  and  the  results  tracked 
over  a  longer  period.  Examining  the  results  of  computer  instruction 
when  imbedded  in  regular  class  activities  would  go  far  in  validating 
or  rejecting  the  results  as  they  occur  in  isolation. 

Limitations 

The  following  limitations  must  be  recognized  in  evaluating  the 
results  of  the  study: 

1.  The  subjects  in  the  current  study  are  an  extremely  homogeneous 
group  of  top-level  high  school  graduates.  They  are  generally 
possessed  of  high  verbal  skills  and  are  also  intimately  familiar 
with  computers.  These  factors  may  serve  to  limit  the  degree  to 
which  one  can  generalize  the  results. 

2.  No  advanced  level  German  students  were  involved  in  the  study. 

3.  Only  one  measure  of  vocabulary  knowledge  was  utilized. 
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4.  Although  most  subjects  seemed  to  take  their  tasks  seriously, 
observational  evidence  (specifically  some  of  the  response  times) 
indicate  that  some  students  did  not  give  the  process  their  full 
attention. 

5.  The  computer  programs  were  rudimentary,  both  in  appearance  and 
function,  due  to  the  limited  programming  skill  of  the 
researcher.  Better  and  more  elaborate  programs  may  well  impact 
the  results  achieved. 

6.  The  research,  though  taking  place  in  the  cadets'  normal  language 
environment,  was  not  derived  from  their  regular  materials.  The 
knowledge  that  they  were  part  of  a  study,  as  well  as  the 
isolation  from  the  normal  flow  of  instruction,  may  have  had  some 
impact  on  the  results.  Ideally,  these  programs  would  evaluate 
the  cadets  performance  within  the  natural  context  of  a  language 
course. 

7.  The  observed  power  in  the  MANCOVA  for  the  interactions  and  the 
covariance  were  low. 
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APPENDIX  A 

RECALL  TEXTS  AND  TRANSLATIONS 
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Batman  —  Aufstand  im  Kinderzimmer 


"Er  soil  ja  besser  sein  als  der  erste",  sagt  die  14jahrige  Maria, 
die  mit  ihrer  Freundin  vor  dem  Loews-Kino  am  Broadway  ansteht  und  nur 
millimeterweise  vorrlickt  in  der  Schlange,  die  sich  vor  der  Kasse 
gebildet  hat.  Obwohl  sie  an  diesem  Wochenende  mithelfen  wird,  einen 
Rekord  zu  brechen,  klingt  sie  nicht  gerade  begeistert.  Es  klingt 
wie:  mitmachen  und  absitzen.  Hier  wird  kein  Fest  angesteuert,  sondern 
eine  Hypnose. 

Fur  Maria  steht  der  Terrain  seit  Wochen  fest,  auf  einem  Plakat, 
drei  Stockwerke  hoch  liber  Times  Square,  schwarz  auf  gelb:  "Batman 
kehrt  zuruck".  Der  Film. 

Batman,  die  Geldmaschine,  spuckt  wieder.  Bereits  im  ersten  Anlauf 
vor  zwei  Jahren  hatte  der  Mann  mit  der  Fledermausmaske  Platz  sechs  in 
der  Liste  der  besten  Filme  aller  Zeit  geschafft.  Nun  spielte  die 
Fortsetzung  schon  am  ersten  Wochenende  46,5  Millionen  Dollar  ein. 
Weltrekord. 

Alle  amerikanische  Kinder  seit  1939  sind  mit  Batman  groft  geworden. 
Der  Fledermaustyp  mit  der  tragischen  Kindheit  ist  ein  schlichterner 
einsamer  Mensch,  der  sich  verwandelt,  wenn  er  sich  die  Maske 
libers tlilpt.  Batman,  tagsliber  braver  Burger,  ist  der  Lotse  durch  die 
Schattenwelt . 
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TRANSLATION:  BATMAN  ARTICLE 
Batman  -  Revolt  in  the  Kids'  Room 

"It  is  supposed  to  be  better  than  the  first,"  says  14  year-old 
Maria,  who,  with  her  friend,  stands  in  front  of  the  Loews  theater  on 
Broadway,  and  only  moves  forward  by  millimeters  in  the  line  that  has 
formed  in  front  of  the  cashier.  Even  though  on  this  weekend  she  will 
help  to  break  a  record,  she  does  not  sound  too  enthusiastic.  It 
sounds  like  "participate  and  sit  it  out."  No  festival  is  being 
celebrated  here,  rather  a  hypnosis. 

For  Maria  the  date  was  fixed  for  weeks,  on  a  placard  three 
stories  high  above  Times  Square,  black  on  yellow:  "Batman  Returns." 
The  film. 

Batman,  the  money  machine  spits  again.  Already  in  the  first 
run  two  years  ago  did  the  man  in  the  batmask  achieve  sixth  place  in 
the  list  of  the  best  movies  of  all  time.  Now  the  continuation  has 
already  produced  $46.5  million  on  the  first  weekend. 

All  American  children  since  1939  have  grown  up  with  Batman. 

The  Bat-figure  with  the  tragic  childhood  is  a  shy,  lonely  person  who 
transforms  himself  when  he  dons  the  batmask.  Batman,  by  day  good 
citizen,  is  the  pilot  through  the  shadow  world. 
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Prof.  Dr.  E  Buchter-Bernhardt 

227  Arps  Hall 

1945  N.  High  Street 

The  Ohio  State  University 

Columbus,  OH  43210 

USA 

Liebe  Frau  Buchter-Bernhardt, 

in  der  Anlage  finden  Sie  die  Dinge,  die  ich  Ihnen  in  Newark 
versprochen  habe.  Wenn  Sie  an  dem  einen  odern  andern  von  uns 
interessiert  sein  sollten,  konnen  wir  dies  gerne  kopieren. 

Unnotig  zu  sagen,  daft  es  groften  Spaft  gemacht  hat,  Sie  kennenzulernen, 
mit  Ihnen  zu  plaudernund  gemeinsame  Interessen  und  Bekannte  zu 
entdecken. 

Ob  Sie  so  nett  sein  konnten,  mir  bei  Gelegenheit  den  Namen  und  die 
Adresse  Ihres  Mitarbeiters ,  der  jetzt  in  Virginia  ist,  mitzuteilen, 
damit  ich  auch  ihm  die  versprochenen  Materialien  schicken  kann.  Ich 
vergaft,  mir  seine  Adresse  auf zuschreiben. 

Mit  den  besten  Gruften  und  alien  guten  Wunschen  bin  ich 

Ihr 

Dr.  H.  Schwarz 
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TRANSLATION:  BERNHARDT  LETTER 


Prof.  Dr.  E.  Buchter-Bernhardt 

227  Arps  Hall 

1945  N.  High  Street 

The  Ohio  State  University 

Columbus,  OH  43210 

USA 

Dear  Frau  Buchter-Bernhardt 

In  the  enclosure  you  will  find  the  things  that  I  promised  you  in 
Newark.  If  you  should  be  interested  in  one  or  the  other  from  us,  we 
can  gladly  copy  that. 

Needless  to  say,  it  was  a  lot  of  fun  to  meet  you,  chat  with  you,  and 
discover  common  interests  and  acquaintances. 

If  you  would  be  so  kind,  when  you  have  an  opportunity,  to  send  me  the 
names  and  addresses  of  your  colleague  who  is  now  in  Virginia,  so  that 
I  can  send  him  the  materials  that  I  promised  him.  I  forgot  to  write 
his  address  down. 

With  the  best  greeting  and  good  wishes,  I  am 
Your, 

Dr.  H  Schwarz 
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Auf  Die  Schnelle  In  Die  Feme  —  Erst  Packen  Dann  Buchen 


So  schnell  kann  es  gehen:  Am  Dienstag  letzer  Woche  dachte  Peter 
Frisch  noch  dariiber  nach,  ob  er  sich  einen  Trip  nach  Spanien  leisten 
konnte.  Am  Donnerstag  jettete  er  dann  doch  lieber  nach  San 
Franzisko.  895  Mark  furs  Ticket  nach  Kalifornien  und  zuruck  — 
dieses  Angebot  hatte  den  Munchner  nicht  lange  zogern  lassen.  Muft  man 
vielleicht  mit  einer  Stewardess  verlobt  sein,  um  so  billig  um  die 
halbe  Welt  zu  jetten?  Des  Ratsels  Losung  ist  viel  einfacher:  Als 
den  Munchner  das  Fernweh  uberkam,  hatte  er  sich  bei  den  Last-Minute- 
Buros  umgehort.  Bei  der  Tonband-Ansage  von  Lf Tours  wurde  er  f undig. 

Noch  rascher  ging's  beim  Munchner  Studenten  Manfred  Kanzler:  er 
packte  einfach  Zahnbiirste  und  Scheckbuch  ein  und  fuhr  zum  Flughafen. 
Da  hatte  er  noch  keine  Ahnung,  wohin  die  Reise  gehen  sollte.  Drei 
Stunden  spater  saft  er  schon  im  Jet  nach  Eliat  am  Roten  Meer  —  fur 
498  Mark. 

Fur  Verkauferin  Beate  Baskos  vom  ABR- Last-Minute-Service  am  Flughafen 
ist  das  nichts  Ungewohnliches :  Sie  vermittelt  jedes  Wochenende 
Feriengluck  gleich  dutzendweise  in  letzter  Minute.  Der  Schlufl- 
Verkauf  von  Urlaubsreisen,  vor  drei  Jahren  noch  fast  unbekannt, 
erhebt  jetzt  den  groBen  Boom. 
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TRANSLATION:  TRAVEL  ARTICLE 

In  a  Hurry  into  the  Distance  -  Pack  First  then  Book 

This  is  how  fast  it  can  happen:  On  Tuesday  last  week  Peter 
Frisch  was  still  thinking  if  he  could  afford  a  trip  to  Spain.  On 
Thursday  he  flew,  instead,  to  San  Francisco.  895  Marks  for  a  ticket 
to  California  and  back  —  the  Munich  student  could  not  hesitate  long. 
Does  one  perhaps  have  to  be  engaged  to  a  Stewardess  in  order  to  fly 
so  cheaply  half-way  around  the  world?  This  puzzle's  solution  is  much 
simpler:  As  the  man  from  Munich  was  overcome  with  a  yearning  to 

travel,  he  checked  out  the  last-minute  agencies.  On  the  tape 
recorded  messages  of  L' Tours  he  struck  pay  dirt. 

It  went  even  faster  for  the  Munich  student  Manfred  Kanzler:  he 
simply  packed  a  toothbrush  and  a  checkbook  and  drove  to  the  airport. 
There,  he  still  had  no  idea  where  the  trip  would  go  to.  Three  hours 
later  he  already  sat  in  a  jet  to  Eliat  on  the  Red  Sea  for  498  Marks. 
For  the  travel  agent  Beate  Baskos  of  ABR  Last  Minute  Service  located 
at  the  airport,  that  is  nothing  unusual:  she  arranges  dream 
vacations  at  the  last  minute  by  the  dozen  every  weekend.  The  close 
out  sale  of  travel  packages,  still  unkown  three  years  ago,  has  now 
risen  to  a  great  boom. 
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APPENDIX  B 
TABULATED  DATA 
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Comments  on  Tabulated  Data 

Although  the  following  table  is  fairly  intuitive,  several  items 
require  additional  explanation: 

Under  delayed  response  time  those  numbers  in  brackets  [  ]  represent 

those  subjects  who  took  the  delayed  test  at  a  slightly  different  time 
than  the  remainder  of  the  cadets.  In  all  cases  these  tests  were 
administered  within  two  days  of  the  primary  delayed  test.  The  cells 
with  no  entries  represent  those  subjects  that  were  unable  to  take  the 
delayed  test. 

Under  the  SAT  scores,  the  scores  with  an  asterisk  represent 
conversions  from  the  ACT. 

Under  the  category  of  words  accessed,  the  first  number  indicates  the 
total  number  of  words  accessed  and  the  number  in  parentheses 
represents  the  number  of  target  words  accessed. 
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Tabulated  Data 


Subject 

Coded  ID 

Response  times  (immediate) 

Response  times  (delayed) 

Word  knowledge  (immediate) 

Word  knowledge  (delayed) 

Recall  scores  (raw) 

Batman  article 

Letter 

Travel  article 

SAT  score 

ACT  english 

ACT  reading 

Avg  Reading  Time  (secs) 

#  Words  accessed: 

Batman  Article 

Letter 


Total 

! 

Total  Target  Words 

Tabulated  Data 


Subject 

8 

9 

10 

11 

12 

13 

LJ4_ 

| 

GSH 

GGU 

GHA 

■ctsf 

18664 

19674 

12595 

14642 

19602 

15956 

14599 

Response  1 

times  (immediate) 

5.02 

4.49 

10.23 

7.11 

7.58 

7.35 

Response  times  (delayed) 

2.14 

5.97 

7.36 

12.25 

6.75 

Word  knowledge  (immediate) 

7 

9 

3 

io 

6 

11 

9 

Word  knowledge  (delayed) 

1 

10 

6 

2 

3 

Recall  scores  (raw) 

Batman  article 

37 

64 

36 

36 

30 

21 

10 

Letter 

22 

19 

6 

22 

Travel  article 

34 

18 

24 

24 

11 

Totals 

89 

76 

79 

92 

51 

43 

Kil*J 

25.33 

17 

14.33 

■ 

SAT  score 

IKEE1 

530 

550 

*570 

600 

ACT  english 

25 

ACT  reading 

26 

28 

Avg  Reading  Time  (secs) 

340 

451 

233 

#  Words  accessed: 

Batman  Article 

(Letter 

Travel  Article 

Total 

Total  Target  Words 
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Tabulated  Data 


Subject 

15 

wm 

17 

18 

19 

20 

21 

nawi 

Coded  ID 

GTH 

a 

GCO 

GJE 

GKE 

GLO 

11818 

11579 

12318 

14313 

15409 

16422 

12556 

Response  1 

imes  (immediate) 

5.33 

7.31 

13.07 

5.19 

7.64 

WE3 

Response  times  (delayed) 

5.8 

4.55 

4.54 

12.68 

4.62 

[5.60] 

4.12 

Word  knowledge  (immediate) 

8 

1 

13 

9 

16 

m 

5 

Word  knowledge  (delayed) 

1 

5 

5 

9 

r~ — 

Recall  scores  (raw) 

Batman  article 

24 

45 

36 

49 

43 

18 

61 

Letter 

1— 

17 

41 

22 

47 

WKal 

WIH 

22 

Travel  article 

33 

34 

56 

Hi 

WES 

19 

Totals 

58 

119 

92 

152 

HB3 

84 

102 

19.33 

39.67 

30.67 

50.67 

49.33 

28 

34 

SAT  score 

*610 

*630 

660 

660 

ACT  english 

27 

28 

ACT  reading 

36 

26 

29 

— 

HWi 

■Hi: 

Avg  Reading  Time  (secs) 

414 

501 

349 

#  Words  accessed: 

Batman  Article 

Letter 

lEM 

Total 

Total  Target  Words 
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Tabulated  Data 


Subject 

22 

23 

24 

25 

26 

27 

28 

Coded  ID 

GBL 

GCA 

GCI 

GWI 

GIN 

GME 

GCO 

12023 

18051 

15611 

12153 

15785 

15321 

13550 

■M 

Response  1 

imes  (immediate) 

7.54 

3.53 

KE2 

4.69 

■ED 

6.29 

Response  times  (delayed) 

7.41 

■a 

6.27 

4.95 

1 

■ 

Word  knowledge  (immediate) 

15 

0 

7 

9 

■ED 

17 

27 

Word  knowledge  (delayed) 

9 

0 

11 

5 

■D 

17 

14 

— 

Recall  scores  (raw) 

Batman  article 

8 

121 

34 

331 

32 

35 

■S 

5 

24 

21 

_ 20j 

12 

B9BH 

wm a 

25 

21 

54 

25 

16 

Totals 

■Q31 

38 

57 

109 

78 

60 

60.33 

12.67 

19 

36.33 

26 

20 

SAT  score 

640 

*520 

690 

ACT  english 

22 

IKS! 

ACT  reading 

21 

HE3 

1 

Avg  Reading  Time  (secs) 

325 

198 

296 

!KSI 

383 

— 

#  Words  accessed: 

immESEmm 

n 

Travel  Article 

Total 

Total  Target  Words 
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Tabulated  Data 


sM* 


Subject 

29 

30 

31 

32 

33 

34 

35 

jjjjjjjiM 

Coded  ID 

GER 

GLA 

GPA 

14191 

18925 

18498 

11585 

11650 

19991 

11325 

Response  1 

times  (immediate) 

5.72 

mm 

mm 

1H 

mBa 

■Hal 

Response  times  (delayed) 

KEE 

MSB 

tiw.ni 

SIS 

Word  knowledge  (immediate) 

18 

40 

44 

Word  knowledge  (delayed) 

9 

29 

9 

35 

9 

Recall  scores  (raw) 

PPM 

■ 

Batman  article 

23 

■HE] 

^El 

Letter 

■no 

Travel  article 

mm 

m 

9 

EBS9MI 

38 

mmn 

44 

71 

103 

40 

MgjMjlgg 

' 

mm 

14.67 

23.67 

34.33 

13.33 

■ 

SAT  score 

660 

690 

560 

*610 

*630 

590 

ACT  english 

27 

ACT  reading 

H 

1 

■mwk 

Avg  Reading  Time  (secs) 

mm 

mm 

540 

179 

#  Words  accessed: 

Batman  Article 

Letter 

Travel  Article 

Total 

Total  Target  Words 
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Tabulated  Data 


Subject 

36 

37 

38 

39 

40 

41 

42 

GDI 

GMA 

GOT 

GDA 

GLA 

GCO 

GKI 

BHBH 

11976 

17836 

13779 

16187 

18631 

11050 

12189 

USUI 

Response  1 

times  (immediate) 

5.9 

5.08 

ir&i 

7.14 

7.06 

Response  times  (delayed) 

3.9 

6.51 

6.47 

[6.37] 

4.66 

6.59 

10.07 

■HI 

Word  knowledge  (immediate) 

30 

29 

33 

33 

34 

33 

Word  knowledge  (delayed) 

20 

24 

19 

[24] 

24 

29 

— 

Batman  article 

35 

24 

45 

29 

38 

48 

54 

naHi 

15 

23 

21 

19 

25 

HEei 

25 

26 

35 

27 

35 

34 

■a 

36 

Totals 

76 

82 

93 

83 

97 

134 

115 

38.33 

SAT  score 

660 

560 

*630 

600 

600 

*650 

ACT  english 

Si 

28 

SSI 

29 

ACT  reading 

33 

27 

i 

HHI 

Avg  Reading  Time  (secs) 

327 

161 

411 

224 

490 

KEE] 

607 

#  Words  accessed: 

Batman  Article 

Letter 

Travel  Article 

Total 

Total  Target  Words 
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Tabulated  Data 


Subject 

43 

44 

45 

46 

47 

48 

49 

Coded  ID 

GLI 

GSC 

GSE 

GAD 

GEL 

19771 

ieei-EI 

B 

15293 

17768 

12057 

14403 

mHm, 

Response  1 

imes  (immediate) 

8.08 

4.69 

MM 

3.76 

■OH 

7.13 

Response  times  (delayed) 

6.52 

[10.59] 

WEXSk 

B 

5.23 

5.92 

1 

■Ml 

Word  knowledge  (immediate) 

46 

35 

HIS 

mm 

Word  knowledge  (delayed) 

35 

[14] 

MM 

■1 

Recall  scores  (raw) 

■ 

Batman  article 

48 

44 

56 

17 

23 

WMB 

23 

Letter 

24 

26 

44 

19 

36 

30 

Travel  article 

42 

26l 

18 

■HI 

29 

50 

Totals 

112 

126 

54 

113 

103 

38 

37.33 

42 

18] 

WBSM 

SAT  score 

■■i 

620 

*630 

610 

610 

*570 

ACT  english 

■■ 

28 

25 

ACT  reading 

27 

29 

Avg  Reading  Time  (secs) 

402 

525 

482 

#  Words  accessed: 

Batman  Article 

Letter 

■  ■ 

DSBHH 

IHI 

Total 

Total  Target  Words 
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Tabulated  Data 


Subject 

50 

51 

52 

53 

54 

55 

56 

1BSBM 

GZE 

GDO 

GFE 

GWA 

GAS 

GHA 

ipg| 

18703 

19917 

17437 

15494 

12607 

Response  1 

times  (immediate) 

6.61 

7.04 

5.82 

7.99 

7.01 

7.32 

Response  times  (delayed) 

7.85 

7.71 

6.96 

6.73 

14.35 

5.54 

Word  knowledge  (immediate) 

25 

7 

MB 

12 

13 

5 

9 

Word  knowledge  (delayed) 

20 

mm 

15 

11 

6 

14 

Recall  scores  (raw) 

Batman  article 

33 

29 

47 

50 

51 

26 

16 

mm 

37 

48 

48 

43 

23 

mmm 

■Q 

6 

37 

50 

33 

22 

MKE3 

Totals 

— jgra 

72 

132 

148 

127 

71 

39 

24 

44 

23.67 

13 

■  ■ 

SAT  score 

590 

650 

640 

*610 

580 

*630 

590 

ACT  english 

27 

28 

ACT  reading 

27 

26 

|Avg  Reading  Time  (secs) 

243 

496 

655 

535 

375 

539 

350 

#  Words  accessed: 

Batman  Article 

29  (3) 

38(4) 

13(2) 

■n 

4(1) 

2 

Letter 

24(6) 

mm 

8 

16(1) 

10(1) 

(Travel  Article 

3 

28(6) 

9(2) 

20  (1) 

5 

Total 

46 

98 

40 

17 

Total  Target  Words 

9 

16 

8 

2 

3 

1 

124 


Tabulated  Data 


Subject 

57 

58 

59 

60 

61 

62 

63 

GHO 

EEZ3I 

GSC 

GHE 

GMA 

GBR 

14041 

mm 

16730 

17212 

19837 

19631 

■Ml 

Response  1 

times  (immediate) 

8.73 

3.03 

6 

4.59 

8.52 

Response  times  (delayed) 

4.86 

2.6 

[5.21] 

7.06 

— 

■ 

Word  knowledge  (immediate) 

6 

0 

7 

Word  knowledge  (delayed) 

4 

3 

8 

6 

7 

■Q 

Recall  scores  (raw) 

Batman  article 

34 

24 

34 

HES 

warn 

36 

Letter 

4 

18 

30 

25 

21 

KQ 

20 

Travel  article 

7 

22 

27 

11 

22 

25 

Totals 

45 

80 

81 

70 

83 

81 

27 

49 

27 

SAT  score 

*650 

*800 

*630 

600 

ACT  english 

■ 

29 

36 

28 

ACT  reading 

WSl 

■kei 

32 

36" 

32 

■ 

Avg  Reading  Time  (secs) 

704 

240 

445 

553 

675 

572 

605 

#  Words  accessed: 

Batman  Article 

12 

3 

20  (1) 

6(1) 

17(2) 

25  (1) 

12(2) 

Letter 

16(1) 

5(1) 

16(4) 

8(3) 

18(2) 

21  (6) 

18(4) 

Travel  Article 

20(2) 

7(1) 

12(1) 

24(3) 

10(2) 

Total 

40 

8 

56 

21 

47 

70 

40 

Total  Target  Words 

1 

1 

7 

5 

5 

10 

8 

125 


Tabulated  Data 


Subject 

65 

66 

68 

69 

70 

GCO 

GFI 

GSC 

GLY 

Bfgpgplf 

16302 

12496 

13303 

17849 

Response  1 

times  (immediate) 

4.98 

10.13 

8.46 

4.19 

7.99 

8.35 

Response  times  (delayed) 

2.39 

5.03 

11.58 

7.66 

4.88 

7.46 

9.59 

Word  knowledge  (immediate) 

6 

14 

13 

19 

13 

■B 

24 

Word  knowledge  (delayed) 

0 

9 

11 

16 

15 

ii 

26 

Recall  scores  (raw) 

Hi 

Batman  article 

49 

K1 

52 

32 

H 

74 

Letter 

34 

■ED 

45 

WMM 

ME! 

51 

Travel  article 

■ 

32 

mm 

31 

mm 

■ED 

58 

Totals 

48 

HQ 

135 

128 

141 

■ 

183 

45 

42.67 

47 

61 

WMSB 

■SMB1 

*590 

*570 
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Recall  Protocol  for  Batman  Article 


«GC01S685  [  625.9316: 0]»  He  should  be  better  than  the  first,  says  the  14  year  old  Maria  as  she 
stands  in  the  theater  on  Broadway. 

American  kids  have  loved  batman  since  1939.  The  sequel  made  46.5  million  dollars. 


«GC0186S6» 

//14jaehrigc{19.06055}//absitzcn{l  11.11 13}//alier{  1 17.541}//angcstcuert{  124.95 12}//Anlauf{  13 1.541}// 

ansteht{135.1699}//begeistcrt{l43.6309}//soll{162.7402}//gebildet{229.4199}//Kasse{243.8711}//klingt 

{286.2109>//angesteuert{316.S613}//Fest{334.3301}//soll{427.541}// 
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Recall  Protocol  for  Bernhardt  Letter 


u  60 

«GC0186S6  [668.0503:  197.3496]»  Address 
Dear, 

Enclosed,  you  will  find  the  promised  materials.  If  you  should  need,  we  can  make  another  copy. 

Needless  to  say,  it  is  very  fun  and  you  can  meet  people  to  converse  with  who  share  your  same  interests. 
You  can  send  it  to  your  colleagues. 

I  hope  all  is  well. 

yours, 
some  name 


«GC01S686» 

//Anlage{2S.56055}//Dingc{42.56055}//vcrsprochencn{66.17969}//odcrn{100.2402}//vcrsprochenen{12 
9.8906}//intci;cssicrt{  143. 2402  }//solltcn{  157.8496}//andcrn{  178.2793  }//kopiercn{205.6895}//Ihncn{3 17 
.3008}//p!audcm{339.2695}//gcmcinsamc{391.7207}//Bckanntc{412.5391)//entdccken{426.709}//Licbc 
{493.2793}//Mitarbcitcrs{531.7305}//mitzuteilcn{545.4102}//schicken{566.2207}/AVuenschen{618.400 
4}//entdcckcn{649}// 


Recall  Protocol  for  Travel  Article 


18(0' 


«GC018oSS  [  776.4805:  197.3496]»  Make  a  fast  trip.  Make  reservations 

So  fast  can  it  go.  On  Tuesday  last  week,  Peter  Frisch  wanted  to  fly  to  Spain.  On  the  next  Thursday,  he 
would  love  to  fly  to  San  Francisco.  895  mark  for  a  trip  to  California  and  back.  This  offer  doesn’t  last  very 
long!  Perhaps  you  have  to  be  engaged  with  a  stewardess  to  fly  halfway  around  the  world  for  so  cheap. 


«GC01S686» 

//Auf{15.38086}//Schnelle{28.17969}//Feme{41.36133}//Ferne{49.21094}//darueber{92.660l6}//nach{ 
1 10.0703 }//darueber{145.3906}//darueber{202.791}//Angebot{224.7012}//Raetsels{263.4805}//ABR- 

Last-Minute- 

Service{328.8906}//So{386. 1797}//dachte{446. 1113  }//darueber{46 1 ,4297}//jctten{480.2 109}//jettete{48 
5.0996}//Buchen{765.0605}// 


Immediate  Knowledge  Test 


GOO 

348554»  1  not  important»  2  Don’t  really  know»  3  Never  heard  of  it»  4  puzzlingness.  »  5  a 
property  of  sound  or  light 

??»  6  lethargic;  sluggish»  7  faster, quicker»  8  cngagcd»  9  engaged»  10  acquaintences 
??»  11  together»  12  can't  remember 

»  13  to  converse  \vith»  14  about»  15  acquainlenccs»  16  spits»  17  together»  18  Puzzle»  19 
shadow  \vorld»  20  unnecessary»  21  sequel»  22  can't  remember»  23  about 
??»  24  can’t  remember 


348554<>»  1  4.71999999999753oKnown»  2  5.99000000000 16oFamiliar»  3 
3,8399999999965  l<>Unknown>>  4  3.0799999999981  !<>Known>>  5 
7.73999999999796<>Unknown»  6  7.5S000000000175<>Unknown»  7 

9.12000000000262oKno\vn»  8  7.090000000000 15<>Familiar»  9  7.40999999999985<>Known»  10 
5.86999999999398oFamiliar»  1 1  5.33000000000 175<>Kno\\Ti»  12 
9.88000000000 102<>Familiar»  13  5.220000000001 16<>Known»  14 
8.5699999999997 l<>Familiar»  15  4.45000000000073<>Familiar»  16 

2.85999999999694<>Known>>  17  6.25999999999S4oFamiliar»  IS  6.63999999999942oKnown» 
19  2.63999999999942<>Kno\\m»  20  1.97999999999956<>Known»  21 

3.29000000000087oKnown»  22  5.88000000000 102<>Familiar»  23 

ll.l99999999997loFamiliar»  24  13.24000000000 16<>Familiar»  24  13.24000000000 16<>FamiIiar 


34S554o«»  0»  NO  ANS«»  1»  I  don’t  know  what  the  study  was  trying  to  accomplish,  so  I  can’t 
really  give  my  impression  as  to  its  effectiveness.  All  I  can  say  is  that  the  I  only  got  a  little  bit  of  each, 
passage  translated.  The  vocabulary  was  easily  learned,  but  I  didn't  retain  it  very  wcll.<o>  2»  Possibly, 
if  the  words  were  reviewed  over  and  over.  Just  one  time  memorizing  them  is  not  effective  in  the  long 
run.<o>  3»  It  was  difficult.  Almost  to  the  point  where  I  stopped  trying.  <o>  4»  Reviewing  is  a 
necessity  if  the  vocabulary  program  is  to  be  effective.  Not  just  reviewing  the  same  day,  but  on  separate 
occasions  <o>  5»  Maybe  a  little  bit.«»  6»  I  retained  a  more  than  half  of  the  words  long  term. 


348554 

(\M)  ( S.c.6) 
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Delayed  Knowledge  Test 


S^S"on*tknow>>  4  a»  5  can,  ^ 

know>>  9  yo  no  se»  10  involvcd»  1 1  don't  kno\v»  12  spit»  13  7»  14  7»  15  7»  16 
acquaintance»  17  ?»  IS  ? 


34S554o»  1  4  56000000000 13  l<>FamiUar»  2  5.329999999998 lloKno\vn»  3 
5  03009909999S69oUnknown»  4  10.l599999999999<>FamiUar»  5  ■  . 

17.360b000000006oFamiliar»  6  4.38999999999942oFamUiat»  7  3.61999999999893  Known 
3.459999999999 13<>Unknon*n»  9  7.68999999999$69<>Familiar»  10 
11  6399999999994 oFamiliar»  11  7.36000000000058oFanuliar»  12 
3.350000000002 18oKno\vn»  13  8.13000000000 102 oFamlliar»  14 
8.23999999999790<>Famlliar»  15  5.94000000000233 oFamiUar»  lo 
4.40000000000 1 46<>Kno\vn»  17  4.77999999999S34oFamaia:^>  IS 
4,060000000001 3 l<>Unkno\vn»  18  4.06000000000 131 oUnknown 

i  . 
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1.  plaudern 

Unknown 

□  Partial 

2.  Schattenwelt 

0  Unknown 

□  Partial 

o 

3.  gemeinsame 

pf  Unknov/n 

□  Partial 

£ 

4.  Ratsel 

0  Unknown 

□  Partial 

'r 

5.  Fortsetzung 

0  Unknown 

Q  Partial 

e 

6.  rascher 

0' Unknown 

□  Partial 

7 .  unndtig 

[0  Unknown 

□  Partial 

u> 

8 .  bereits 

(2f  Unknown 

□  Partial 

9.  entdecken 

0"  Unknown 

□  Partial 

10.  verlobt 

pf  Unknown 

□  partial 

11.  ungewShnlich 

[2f  Unknown 

0  Partial 

lA 

12.  spuckt 

Q  Unknown 

0  Partial 

13.  Gelegenheit 

0"  Unknown 

0  Partial 

r=- 

14.  verwandelt 

0  Unknov/n 

0  Partial 

15.  vermittelt 

tZf  Unknown 

0  Partial 

K 

16.  Bekannte 

[  |  Unknown 

0  Partial 

17.  Angebot 

0^  Unknown 

0  Partial 

0 

18.  begeistert 

(0  Unknown 

0  Partial 

□  Known  I 
\7f  Known  3 
Q  Known 

□  Known  | 

□  Known  | 

□  Known  | 

[ 2f  Known  3 

□  Known 

□  Known  \ 

□  Knov/n  \ 

□  Knov/n  \ 
Knov/n  3 

□  Knov/n  1 

□  Known  V 

□  Knov/n  ' 
QK  Knov/n 

□  Known 

□  Knov/n 
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List  of  Instructed  Words 


Focus  Group 

Non* 

-Focus  Group 

1. 

plaudern 

1. 

schicken 

2. 

Schattenwelt 

2. 

Fernweh 

3. 

geme insame 

3. 

Ahnung 

4. 

Ratsel 

4. 

Wanderlust 

5. 

Fortsetzung 

5. 

obwohl 

6. 

rascher 

6. 

einfacher 

7 . 

unnotig 

7. 

spater 

8. 

bereits 

8 . 

Dinge 

9. 

entdecken 

9. 

Plakat 

10. 

verlobt 

10. 

mitmachen 

11. 

ungewohnlich 

11. 

Anlage 

12. 

spuckt 

12. 

nett 

13. 

Gelegenheit 

13. 

kopieren 

14. 

verwandelt 

14. 

buchen 

15. 

vermittelt 

15. 

Termin 

16. 

Bekannte 

16. 

Verkauferin 

17. 

Angebot 

17. 

versprochen 

18. 

begeistert 

18. 

braver 
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